[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:53 dist-test-slave-nj21 krb5kdc[4303](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:53 dist-test-slave-nj21 krb5kdc[4303](info): set up 2 sockets
Aug 11 12:08:53 dist-test-slave-nj21 krb5kdc[4303](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 12:08:54.339210 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.377s user 0.001s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:08:54 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:08:54.521183 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45235
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45235
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:54.895776 4328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:54.896220 4328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:54.896369 4328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:54.903435 4328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:54.903538 4328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:54.903623 4328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:54.903724 4328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:54.903788 4328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:54.913656 4328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45235
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:54.915573 4328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:54.917452 4328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:54.935323 4339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:54.935905 4345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:54.935936 4340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:54.936530 4328 server_base.cc:1034] running on GCE node
I20260811 12:08:54.937601 4328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:54.939888 4328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:54.941125 4328 hybrid_clock.cc:648] HybridClock initialized: now 1786450134941033 us; error 102 us; skew 500 ppm
I20260811 12:08:54.941687 4328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:54 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450134, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:54.945189 4328 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:08:54.947582 4328 webserver.cc:460] Webserver started at http://127.4.48.254:36433/ using document root <none> and password file <none>
I20260811 12:08:54.948704 4328 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:54.948853 4328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:54.949326 4328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:54.953406 4328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "701904cb22a84b64b7d971584ecf58d7"
format_stamp: "Formatted at 2026-08-11 12:08:54 on dist-test-slave-nj21"
server_key: "fdfcde9a4cf588f112ccdeca04b8583a"
server_key_iv: "0a71292393ea6c2ba52c087ff620d4a3"
server_key_version: "encryptionkey@0"
I20260811 12:08:54.954252 4328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "701904cb22a84b64b7d971584ecf58d7"
format_stamp: "Formatted at 2026-08-11 12:08:54 on dist-test-slave-nj21"
server_key: "fdfcde9a4cf588f112ccdeca04b8583a"
server_key_iv: "0a71292393ea6c2ba52c087ff620d4a3"
server_key_version: "encryptionkey@0"
I20260811 12:08:54.962571 4328 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260811 12:08:54.968236 4364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:54.970188 4328 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:08:54.970448 4328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "701904cb22a84b64b7d971584ecf58d7"
format_stamp: "Formatted at 2026-08-11 12:08:54 on dist-test-slave-nj21"
server_key: "fdfcde9a4cf588f112ccdeca04b8583a"
server_key_iv: "0a71292393ea6c2ba52c087ff620d4a3"
server_key_version: "encryptionkey@0"
I20260811 12:08:54.970651 4328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:55.006770 4328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:55.011879 4328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:55.012230 4328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:55.035512 4328 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45235
I20260811 12:08:55.035598 4460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45235 every 8 connection(s)
I20260811 12:08:55.037921 4328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:55.044960 4461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:55.046466 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4328
I20260811 12:08:55.046901 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:08:55.050053 4291 external_mini_cluster.cc:1428] Setting key d7d6f4b066dfa2db38e6f4e02e927210
I20260811 12:08:55.066430 4461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: Bootstrap starting.
I20260811 12:08:55.073549 4461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:08:55.075524 4461 log.cc:824] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:55.080288 4461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: No bootstrap required, opened a new log
Aug 11 12:08:55 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:08:55.092810 4461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } }
I20260811 12:08:55.093353 4461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:55.093562 4461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 701904cb22a84b64b7d971584ecf58d7, State: Initialized, Role: FOLLOWER
I20260811 12:08:55.094871 4461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } }
I20260811 12:08:55.095237 4461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:55.095551 4461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:55.095798 4461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:55.098919 4461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } }
I20260811 12:08:55.100241 4461 leader_election.cc:302] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 701904cb22a84b64b7d971584ecf58d7; no voters:
I20260811 12:08:55.101358 4461 leader_election.cc:288] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:55.101576 4467 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:55.102563 4465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:55.058953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36560 (local address 127.4.48.254:45235)
0811 12:08:55.060636 (+ 1683us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:55.060648 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:55.060697 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:08:55.073854 (+ 13157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:55.073878 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:55.074525 (+ 647us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:55.074954 (+ 429us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:55.079156 (+ 4202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.080730 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.083376 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.084265 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.084473 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:55.092325 (+ 7852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:55.092390 (+ 65us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:55.092422 (+ 32us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:55.092461 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:55.096144 (+ 3683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:55.097170 (+ 1026us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:55.097176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:55.097181 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:55.097420 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:55.097942 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:55.097945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:55.097947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:55.098662 (+ 715us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:55.098995 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:55.100225 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0811 12:08:55.100559 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1406,"thread_start_us":1057,"threads_started":1}
I20260811 12:08:55.104585 4461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:55.105405 4467 raft_consensus.cc:686] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [term 1 LEADER]: Becoming Leader. State: Replica: 701904cb22a84b64b7d971584ecf58d7, State: Running, Role: LEADER
I20260811 12:08:55.106442 4467 consensus_queue.cc:237] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } }
I20260811 12:08:55.112865 4470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 701904cb22a84b64b7d971584ecf58d7. Latest consensus state: current_term: 1 leader_uuid: "701904cb22a84b64b7d971584ecf58d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } } }
I20260811 12:08:55.112911 4469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "701904cb22a84b64b7d971584ecf58d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "701904cb22a84b64b7d971584ecf58d7" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45235 } } }
I20260811 12:08:55.113407 4470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:55.113505 4469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7 [sys.catalog]: This master's current role is: LEADER
W20260811 12:08:55.122896 4496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:08:55.123080 4496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:08:55.123250 4497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:55.125504 4497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:55.139488 4497 catalog_manager.cc:1347] Generated new cluster ID: 471c27fea0af4f7db032d151ab6c661b
I20260811 12:08:55.139601 4497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:55.153268 4497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:55.154614 4497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:55.169344 4497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 701904cb22a84b64b7d971584ecf58d7: Generated new TSK 0
I20260811 12:08:55.170501 4497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:08:55.229223 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:55.615481 4506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:55.615810 4506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:55.616001 4506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:55.623471 4506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:55.623590 4506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:55.623785 4506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:08:55.633857 4506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:55.635097 4506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:55.636511 4506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:55.650489 4521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:55.650604 4520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:55.650525 4523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:55.651664 4506 server_base.cc:1034] running on GCE node
I20260811 12:08:55.652304 4506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:55.653235 4506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:55.654441 4506 hybrid_clock.cc:648] HybridClock initialized: now 1786450135654386 us; error 40 us; skew 500 ppm
I20260811 12:08:55.654767 4506 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:55 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450135, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:55.658144 4506 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:08:55.659617 4506 webserver.cc:460] Webserver started at http://127.4.48.193:36805/ using document root <none> and password file <none>
I20260811 12:08:55.660315 4506 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:55.660430 4506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:55.660717 4506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:55.663691 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "f7527ffe1f5a449db2e523c34b2abcb9"
format_stamp: "Formatted at 2026-08-11 12:08:55 on dist-test-slave-nj21"
server_key: "056d2f5e42a5245983bceb0cbc0e41af"
server_key_iv: "08509b40283f2e3a4038b9d417a3b3cd"
server_key_version: "encryptionkey@0"
I20260811 12:08:55.664456 4506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "f7527ffe1f5a449db2e523c34b2abcb9"
format_stamp: "Formatted at 2026-08-11 12:08:55 on dist-test-slave-nj21"
server_key: "056d2f5e42a5245983bceb0cbc0e41af"
server_key_iv: "08509b40283f2e3a4038b9d417a3b3cd"
server_key_version: "encryptionkey@0"
I20260811 12:08:55.669862 4506 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:55.673249 4540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:55.674504 4506 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:08:55.674642 4506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "f7527ffe1f5a449db2e523c34b2abcb9"
format_stamp: "Formatted at 2026-08-11 12:08:55 on dist-test-slave-nj21"
server_key: "056d2f5e42a5245983bceb0cbc0e41af"
server_key_iv: "08509b40283f2e3a4038b9d417a3b3cd"
server_key_version: "encryptionkey@0"
I20260811 12:08:55.674850 4506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:55.703824 4506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:55.707993 4506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:55.708287 4506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:55.709473 4506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:08:55.711321 4506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:08:55.711417 4506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:55.711616 4506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:08:55.711702 4506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:55.753501 4506 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:39867
I20260811 12:08:55.753575 4741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:39867 every 8 connection(s)
I20260811 12:08:55.754812 4506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:08:55.756762 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4506
I20260811 12:08:55.756973 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:08:55.757593 4291 external_mini_cluster.cc:1428] Setting key 2f470574688f0e73a996c12696246b85
Aug 11 12:08:55 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450135, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
I20260811 12:08:55.778028 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:55.759759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:48429 (local address 127.4.48.254:45235)
0811 12:08:55.760310 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:55.760315 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:55.761471 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 12:08:55.763489 (+ 2018us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:55.763495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:55.763595 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:55.763717 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:55.765292 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.766038 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.767689 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.768145 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.768270 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:55.772035 (+ 3765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:55.772040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:55.772043 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:55.772076 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:55.774772 (+ 2696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:55.775647 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:55.775651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:55.775654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:55.775819 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:55.776262 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:55.776266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:55.776270 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:55.776560 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:55.776712 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:55.777455 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 12:08:55.777670 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":311,"threads_started":1}
I20260811 12:08:55.779137 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:55.760443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45235 (local address 127.4.48.193:48429)
0811 12:08:55.761330 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:55.761368 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:55.763233 (+ 1865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:55.763970 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:55.763978 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:55.764519 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:55.765122 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:55.765139 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.766192 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.766196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:55.767507 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:55.767529 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:55.768388 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:55.768396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:55.768645 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:55.769301 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:55.769323 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:55.771794 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:55.774906 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:55.774916 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:55.774932 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:55.775536 (+ 604us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:55.775931 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:55.775936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:55.775940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:55.776143 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:55.776683 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:55.776691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:55.777021 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 12:08:55.777313 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 12:08:55.777879 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":401,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
I20260811 12:08:55.780845 4742 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45235
I20260811 12:08:55.781373 4742 heartbeater.cc:464] Registering TS with master...
I20260811 12:08:55.782230 4742 heartbeater.cc:511] Master 127.4.48.254:45235 requested a full tablet report, sending...
I20260811 12:08:55.786127 4396 ts_manager.cc:194] Registered new tserver with Master: f7527ffe1f5a449db2e523c34b2abcb9 (127.4.48.193:39867)
I20260811 12:08:55.788354 4396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:48429
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:08:55.802099 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:56.184587 4755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:56.184871 4755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:56.184990 4755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:56.192474 4755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:56.192584 4755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:56.192718 4755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:08:56.202772 4755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:56.203984 4755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:56.205317 4755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:56.219172 4766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:56.219210 4773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:56.219288 4767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:56.220386 4755 server_base.cc:1034] running on GCE node
I20260811 12:08:56.220983 4755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:56.221887 4755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:56.223091 4755 hybrid_clock.cc:648] HybridClock initialized: now 1786450136223034 us; error 35 us; skew 500 ppm
I20260811 12:08:56.223395 4755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:56 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450136, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:56.226646 4755 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:08:56.228096 4755 webserver.cc:460] Webserver started at http://127.4.48.194:44363/ using document root <none> and password file <none>
I20260811 12:08:56.228763 4755 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:56.228897 4755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:56.229250 4755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:56.232276 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "354ab0ad500241bdaad6464feb0df789"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "ea8b8bff7b30d097f47453c9d2546916"
server_key_iv: "a7b3688ed0a44ead3a0d67785507e4a2"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.233037 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "354ab0ad500241bdaad6464feb0df789"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "ea8b8bff7b30d097f47453c9d2546916"
server_key_iv: "a7b3688ed0a44ead3a0d67785507e4a2"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.238140 4755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:08:56.241617 4789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:56.242834 4755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:08:56.242995 4755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "354ab0ad500241bdaad6464feb0df789"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "ea8b8bff7b30d097f47453c9d2546916"
server_key_iv: "a7b3688ed0a44ead3a0d67785507e4a2"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.243175 4755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:56.256493 4755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:56.260737 4755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:56.261029 4755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:56.262044 4755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:08:56.263442 4755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:08:56.263532 4755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:56.263664 4755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:08:56.263758 4755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 12:08:56.304203 4755 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:45271
I20260811 12:08:56.304283 4980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:45271 every 8 connection(s)
I20260811 12:08:56.305650 4755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:08:56.309051 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4755
I20260811 12:08:56.309257 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:08:56.309739 4291 external_mini_cluster.cc:1428] Setting key c0a1a1d5511afabdde5e79e3f87e433c
Aug 11 12:08:56 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450136, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:08:56.326634 4984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:56.308218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:32891 (local address 127.4.48.254:45235)
0811 12:08:56.308719 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:56.308725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:56.309974 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 12:08:56.311975 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:56.311981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:56.312061 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:56.312178 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:56.313664 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.314666 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.316270 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.316874 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.316988 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:56.320699 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:56.320704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:56.320707 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:56.320739 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:56.323500 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:56.324237 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:56.324240 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:56.324243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:56.324386 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:56.324917 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:56.324921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:56.324923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:56.325216 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:56.325324 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:56.326177 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 12:08:56.326346 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":286,"threads_started":1}
I20260811 12:08:56.328145 4986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:56.308743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45235 (local address 127.4.48.194:32891)
0811 12:08:56.309797 (+ 1054us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:56.309843 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:56.311722 (+ 1879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:56.312354 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:56.312362 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:56.312893 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:56.313498 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:56.313515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.314782 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.314785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:56.316142 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:56.316154 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.317064 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.317069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:56.317272 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:56.317910 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:56.317932 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:56.320476 (+ 2544us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:56.323629 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:56.323638 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:56.323654 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:56.324115 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:56.324522 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:56.324528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:56.324532 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:56.324808 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:56.325355 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:56.325364 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:56.325801 (+ 437us) client_negotiation.cc:769] Sending connection context
0811 12:08:56.326122 (+ 321us) client_negotiation.cc:240] Negotiation successful
0811 12:08:56.326663 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"thread_start_us":477,"threads_started":1}
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
I20260811 12:08:56.329583 4981 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45235
I20260811 12:08:56.329890 4981 heartbeater.cc:464] Registering TS with master...
I20260811 12:08:56.330710 4981 heartbeater.cc:511] Master 127.4.48.254:45235 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
I20260811 12:08:56.332288 4396 ts_manager.cc:194] Registered new tserver with Master: 354ab0ad500241bdaad6464feb0df789 (127.4.48.194:45271)
I20260811 12:08:56.333169 4396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:32891
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:08:56.352794 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:08:56.792073 4742 heartbeater.cc:503] Master 127.4.48.254:45235 was elected leader, sending a full tablet report...
W20260811 12:08:56.797938 4991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:56.798520 4991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:56.798833 4991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:56.812431 4991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:56.812567 4991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:56.812744 4991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:08:56.823199 4991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45235
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:56.824601 4991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:56.825995 4991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:56.839627 5003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:56.840102 5009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:56.840101 5006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:56.841014 4991 server_base.cc:1034] running on GCE node
I20260811 12:08:56.841595 4991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:56.842499 4991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:56.843732 4991 hybrid_clock.cc:648] HybridClock initialized: now 1786450136843639 us; error 43 us; skew 500 ppm
I20260811 12:08:56.844043 4991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:56 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450136, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:56.847419 4991 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:08:56.848922 4991 webserver.cc:460] Webserver started at http://127.4.48.195:36145/ using document root <none> and password file <none>
I20260811 12:08:56.849587 4991 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:56.849720 4991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:56.850034 4991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:56.852993 4991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "c4718d3d8cd841b5a8344d2d6e94ec4e"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "0c089bff6a2a6b329d62b54c1c1b93be"
server_key_iv: "3b2573e36b26920d87b46d6c644f297a"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.853750 4991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "c4718d3d8cd841b5a8344d2d6e94ec4e"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "0c089bff6a2a6b329d62b54c1c1b93be"
server_key_iv: "3b2573e36b26920d87b46d6c644f297a"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.859000 4991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 12:08:56.862821 5018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:56.864248 4991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:08:56.864413 4991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "c4718d3d8cd841b5a8344d2d6e94ec4e"
format_stamp: "Formatted at 2026-08-11 12:08:56 on dist-test-slave-nj21"
server_key: "0c089bff6a2a6b329d62b54c1c1b93be"
server_key_iv: "3b2573e36b26920d87b46d6c644f297a"
server_key_version: "encryptionkey@0"
I20260811 12:08:56.864599 4991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:56.908622 4991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:56.912883 4991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:56.913219 4991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:56.914309 4991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:08:56.915833 4991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:08:56.915918 4991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:56.916061 4991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:08:56.916131 4991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:56.955807 4991 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44509
I20260811 12:08:56.955874 5226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44509 every 8 connection(s)
I20260811 12:08:56.957089 4991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:08:56.960542 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4991
I20260811 12:08:56.960755 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:08:56.961388 4291 external_mini_cluster.cc:1428] Setting key 2622b1d540004118b7489f663631b994
Aug 11 12:08:56 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450136, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:08:56.978493 5230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:56.959837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35747 (local address 127.4.48.254:45235)
0811 12:08:56.960414 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:56.960421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:56.961585 (+ 1164us) server_negotiation.cc:407] Connection header received
0811 12:08:56.964160 (+ 2575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:56.964166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:56.964253 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:56.964339 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:56.965957 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.966798 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.968450 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.968916 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.969052 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:56.972425 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:56.972440 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:56.972446 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:56.972478 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:56.975156 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:56.975933 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:56.975937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:56.975939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:56.976081 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:56.976530 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:56.976534 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:56.976536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:56.976872 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:56.977034 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:56.977877 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 12:08:56.978091 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":248,"threads_started":1}
I20260811 12:08:56.979422 5231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:56.960443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45235 (local address 127.4.48.195:35747)
0811 12:08:56.961401 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0811 12:08:56.961464 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:08:56.963859 (+ 2395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:08:56.964584 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:08:56.964592 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:08:56.965118 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:08:56.965793 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:56.965812 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.966941 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.966945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:56.968331 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:08:56.968343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:08:56.969126 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:08:56.969131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:08:56.969345 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:56.969946 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:08:56.969968 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:08:56.972250 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:08:56.975317 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:56.975326 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:56.975338 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:56.975815 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:56.976185 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:08:56.976190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:08:56.976194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:08:56.976418 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:08:56.977001 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:08:56.977007 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:08:56.977421 (+ 414us) client_negotiation.cc:769] Sending connection context
0811 12:08:56.977739 (+ 318us) client_negotiation.cc:240] Negotiation successful
0811 12:08:56.978228 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":408,"threads_started":1}
I20260811 12:08:56.980793 5227 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45235
I20260811 12:08:56.981098 5227 heartbeater.cc:464] Registering TS with master...
I20260811 12:08:56.981834 5227 heartbeater.cc:511] Master 127.4.48.254:45235 requested a full tablet report, sending...
I20260811 12:08:56.983318 4396 ts_manager.cc:194] Registered new tserver with Master: c4718d3d8cd841b5a8344d2d6e94ec4e (127.4.48.195:44509)
I20260811 12:08:56.984277 4396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:35747
I20260811 12:08:56.990362 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:08:57.038699 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:57.022454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.4.48.193:39867)
0811 12:08:57.023224 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:57.023232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:57.023257 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:08:57.023316 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:57.023323 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:57.023557 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:57.023726 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:57.025403 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.026483 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.027888 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.028501 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.028611 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:57.032708 (+ 4097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:57.032719 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:57.032733 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:57.032774 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:57.035867 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.036510 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.036517 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.036519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.036664 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.037053 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.037056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.037058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.037423 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:57.037734 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:57.038062 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:08:57.038328 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":372,"threads_started":1}
W20260811 12:08:57.040135 4579 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50912
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:08:57.086796 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:57.072945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.4.48.193:39867)
0811 12:08:57.073127 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:57.073134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:57.073465 (+ 331us) server_negotiation.cc:407] Connection header received
0811 12:08:57.073640 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:57.073644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:57.073742 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:57.073829 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:57.075306 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.076046 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.077296 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.077741 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.077861 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:57.081801 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:57.081806 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:57.081809 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:57.081842 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:57.084324 (+ 2482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.084985 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.084990 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.084992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.085138 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.085482 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.085486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.085488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.085720 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:57.085963 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:57.086250 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 12:08:57.086470 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:08:57.088613 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4506
I20260811 12:08:57.100497 5242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:57.089641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.4.48.193:39867)
0811 12:08:57.089777 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:57.089781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:57.090103 (+ 322us) server_negotiation.cc:407] Connection header received
0811 12:08:57.090206 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:57.090209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:57.090256 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:57.090325 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:57.091414 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.092110 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.093459 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.093963 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.094048 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:57.096166 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:57.096170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:57.096173 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:57.096199 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:57.098492 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.099028 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.099031 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.099033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.099159 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.099432 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.099435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.099437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.099662 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:57.099888 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:57.100061 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:08:57.100202 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:08:57.101186 4737 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:57.188905 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4506
I20260811 12:08:57.212566 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4755
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:08:57.227078 5256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:57.213418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52482 (local address 127.4.48.194:45271)
0811 12:08:57.213972 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:57.213979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:57.214007 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:08:57.214082 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:57.214091 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:57.214384 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:57.214516 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:57.215609 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.216660 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.217788 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.218325 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.218416 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:57.222173 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:57.222185 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:57.222198 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:57.222228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:57.224921 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.225437 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.225444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.225446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.225565 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.225858 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.225860 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.225862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.226174 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:57.226405 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:57.226586 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:08:57.226776 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":274,"threads_started":1}
I20260811 12:08:57.228418 4976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:57.308941 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4755
I20260811 12:08:57.343662 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4991
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[4303](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:08:57.359975 5258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:57.344917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54992 (local address 127.4.48.195:44509)
0811 12:08:57.345643 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:57.345651 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:57.345681 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:08:57.345742 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:57.345748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:57.345978 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:57.346121 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:57.347426 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.348516 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.349785 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:57.350350 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:57.350465 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:57.354600 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:57.354611 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:57.354625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:57.354659 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:57.357201 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.357801 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.357808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.357810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.357938 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:57.358281 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:57.358284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:57.358286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:57.358607 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:57.358914 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:57.359191 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 12:08:57.359451 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":428,"threads_started":1}
I20260811 12:08:57.361258 5218 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:57.493110 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4991
I20260811 12:08:57.515306 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4328
I20260811 12:08:57.516726 4451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:08:57.618782 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4328
2026-08-11T12:08:57Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4719 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[5268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[5268](info): set up 2 sockets
Aug 11 12:08:57 dist-test-slave-nj21 krb5kdc[5268](info): commencing operation
krb5kdc: starting...
W20260811 12:08:58.863308 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:08:58 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:08:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:08:59.039047 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38125
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38125
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:59.368606 5290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:59.368884 5290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:59.368981 5290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:59.375950 5290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:08:59.376041 5290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:59.376093 5290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:59.376148 5290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:08:59.376190 5290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:08:59.385838 5290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38125
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:59.387055 5290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:59.388391 5290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:59.400236 5301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:59.400231 5300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:59.400609 5305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:59.401077 5290 server_base.cc:1034] running on GCE node
I20260811 12:08:59.401763 5290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:59.403052 5290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:59.404285 5290 hybrid_clock.cc:648] HybridClock initialized: now 1786450139404236 us; error 31 us; skew 500 ppm
I20260811 12:08:59.404627 5290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:59 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450139, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:08:59.407548 5290 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:08:59.409016 5290 webserver.cc:460] Webserver started at http://127.4.48.254:38217/ using document root <none> and password file <none>
I20260811 12:08:59.409668 5290 fs_manager.cc:359] Metadata directory not provided
I20260811 12:08:59.409778 5290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:08:59.410089 5290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:08:59.413008 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "d389da5460f149a8ad7f8ef7a2b84499"
format_stamp: "Formatted at 2026-08-11 12:08:59 on dist-test-slave-nj21"
server_key: "567af71cd6f8bcb7f2df398054268d81"
server_key_iv: "7bb302196f491cdc81204167765aa923"
server_key_version: "encryptionkey@0"
I20260811 12:08:59.413745 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "d389da5460f149a8ad7f8ef7a2b84499"
format_stamp: "Formatted at 2026-08-11 12:08:59 on dist-test-slave-nj21"
server_key: "567af71cd6f8bcb7f2df398054268d81"
server_key_iv: "7bb302196f491cdc81204167765aa923"
server_key_version: "encryptionkey@0"
I20260811 12:08:59.419325 5290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 12:08:59.422952 5313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:08:59.424338 5290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:08:59.424499 5290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "d389da5460f149a8ad7f8ef7a2b84499"
format_stamp: "Formatted at 2026-08-11 12:08:59 on dist-test-slave-nj21"
server_key: "567af71cd6f8bcb7f2df398054268d81"
server_key_iv: "7bb302196f491cdc81204167765aa923"
server_key_version: "encryptionkey@0"
I20260811 12:08:59.424665 5290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:08:59.452770 5290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:08:59.457069 5290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:08:59.457351 5290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:08:59.475915 5290 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:38125
I20260811 12:08:59.475951 5409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:38125 every 8 connection(s)
I20260811 12:08:59.477309 5290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:08:59.481535 5411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:08:59.485790 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5290
I20260811 12:08:59.486034 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:08:59.486886 4291 external_mini_cluster.cc:1428] Setting key 7c50dd36fcd2969dd8f513aa7e0ca7ab
I20260811 12:08:59.495738 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499: Bootstrap starting.
I20260811 12:08:59.499908 5411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499: Neither blocks nor log segments found. Creating new log.
Aug 11 12:08:59 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:08:59.501127 5411 log.cc:824] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499: Log is configured to *not* fsync() on all Append() calls
I20260811 12:08:59.503937 5411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499: No bootstrap required, opened a new log
I20260811 12:08:59.509037 5416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:08:59.489094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33604 (local address 127.4.48.254:38125)
0811 12:08:59.489856 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0811 12:08:59.489875 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:08:59.489916 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:08:59.491456 (+ 1540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:08:59.491477 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:08:59.491915 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:08:59.492219 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:08:59.493888 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:59.495183 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:59.496828 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:08:59.497477 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:08:59.497608 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:08:59.502104 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:08:59.502112 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:08:59.502126 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:08:59.502164 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:08:59.505013 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:59.505639 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:59.505646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:59.505652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:59.505829 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:08:59.506180 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:08:59.506184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:08:59.506186 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:08:59.506741 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:08:59.507055 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:08:59.507546 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 12:08:59.507969 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":352,"threads_started":1}
I20260811 12:08:59.511729 5411 raft_consensus.cc:348] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } }
I20260811 12:08:59.512177 5411 raft_consensus.cc:374] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:08:59.512288 5411 raft_consensus.cc:729] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d389da5460f149a8ad7f8ef7a2b84499, State: Initialized, Role: FOLLOWER
I20260811 12:08:59.512873 5411 consensus_queue.cc:260] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } }
I20260811 12:08:59.513125 5411 raft_consensus.cc:388] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:08:59.513283 5411 raft_consensus.cc:482] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:08:59.513432 5411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:08:59.515771 5411 raft_consensus.cc:504] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } }
I20260811 12:08:59.516338 5411 leader_election.cc:302] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d389da5460f149a8ad7f8ef7a2b84499; no voters:
I20260811 12:08:59.516888 5411 leader_election.cc:288] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:08:59.517019 5419 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:08:59.517669 5419 raft_consensus.cc:686] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [term 1 LEADER]: Becoming Leader. State: Replica: d389da5460f149a8ad7f8ef7a2b84499, State: Running, Role: LEADER
I20260811 12:08:59.518244 5419 consensus_queue.cc:237] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } }
I20260811 12:08:59.518841 5411 sys_catalog.cc:564] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:08:59.520941 5421 sys_catalog.cc:455] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d389da5460f149a8ad7f8ef7a2b84499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } } }
I20260811 12:08:59.521267 5421 sys_catalog.cc:458] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:59.522095 5426 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:08:59.521942 5422 sys_catalog.cc:455] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d389da5460f149a8ad7f8ef7a2b84499. Latest consensus state: current_term: 1 leader_uuid: "d389da5460f149a8ad7f8ef7a2b84499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d389da5460f149a8ad7f8ef7a2b84499" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38125 } } }
I20260811 12:08:59.523185 5422 sys_catalog.cc:458] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499 [sys.catalog]: This master's current role is: LEADER
I20260811 12:08:59.527806 5426 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:08:59.537503 5426 catalog_manager.cc:1347] Generated new cluster ID: 023763c5800f414e8e30958301a765ad
I20260811 12:08:59.537608 5426 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:08:59.564352 5426 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:08:59.565654 5426 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:08:59.572111 5426 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d389da5460f149a8ad7f8ef7a2b84499: Generated new TSK 0
I20260811 12:08:59.572888 5426 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:08:59.622504 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:08:59.943281 5450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:08:59.943539 5450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:08:59.943688 5450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:08:59.950868 5450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:08:59.950966 5450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:08:59.951108 5450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:08:59.960451 5450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:08:59.961745 5450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:08:59.963088 5450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:08:59.988518 5464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:59.989318 5467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:08:59.988592 5463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:08:59.990368 5450 server_base.cc:1034] running on GCE node
I20260811 12:08:59.991338 5450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:08:59.992380 5450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:08:59.995937 5450 hybrid_clock.cc:648] HybridClock initialized: now 1786450139995816 us; error 156 us; skew 500 ppm
I20260811 12:08:59.996253 5450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:08:59 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450139, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:00.001667 5450 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:00.005298 5450 webserver.cc:460] Webserver started at http://127.4.48.193:33317/ using document root <none> and password file <none>
I20260811 12:09:00.006031 5450 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:00.006150 5450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:00.006750 5450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:00.012984 5450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "2186a8103a2645ffa79f40224503db26"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "484ca264370b49eaa6be4ff95d16e410"
server_key_iv: "d2ede0813bcfa4f1e1e699709d9c4a8b"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.015372 5450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "2186a8103a2645ffa79f40224503db26"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "484ca264370b49eaa6be4ff95d16e410"
server_key_iv: "d2ede0813bcfa4f1e1e699709d9c4a8b"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.026480 5450 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.007s sys 0.001s
I20260811 12:09:00.034185 5483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.037032 5450 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260811 12:09:00.037205 5450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "2186a8103a2645ffa79f40224503db26"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "484ca264370b49eaa6be4ff95d16e410"
server_key_iv: "d2ede0813bcfa4f1e1e699709d9c4a8b"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.037405 5450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:00.080821 5450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:00.088629 5450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:00.089434 5450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:00.092106 5450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:00.094048 5450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:00.094295 5450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.094923 5450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:00.095198 5450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.131054 5450 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35513
I20260811 12:09:00.131218 5677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35513 every 8 connection(s)
I20260811 12:09:00.132450 5450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:00.141674 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5450
I20260811 12:09:00.141924 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:00.142627 4291 external_mini_cluster.cc:1428] Setting key 6266884e1d2163c08c9465d3773cce3a
Aug 11 12:09:00 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450139, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:00.153302 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:00.135428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:41449 (local address 127.4.48.254:38125)
0811 12:09:00.135907 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:00.135911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:00.136864 (+ 953us) server_negotiation.cc:407] Connection header received
0811 12:09:00.138688 (+ 1824us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:00.138694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:00.138774 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:00.138870 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:00.140407 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.141124 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.142783 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.143179 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.143291 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:00.147041 (+ 3750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:00.147059 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:00.147067 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:00.147110 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:00.150202 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:00.150893 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:00.150898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:00.150901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:00.151052 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:00.151426 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:00.151430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:00.151432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:00.151806 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:00.151955 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:00.152612 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 12:09:00.152860 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":244,"threads_started":1}
I20260811 12:09:00.154184 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:00.136001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38125 (local address 127.4.48.193:41449)
0811 12:09:00.136702 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:00.136744 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:00.138430 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:00.139115 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:00.139123 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:00.139625 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:00.140236 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:00.140254 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.141282 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.141286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:00.142644 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:00.142657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.143408 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.143416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:00.143631 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:00.144372 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:00.144406 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:00.146820 (+ 2414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:00.150337 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:00.150349 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:00.150361 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:00.150776 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:00.151142 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:00.151145 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:00.151147 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:00.151313 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:00.151909 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:00.151915 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:00.152219 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 12:09:00.152477 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 12:09:00.152969 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":269,"threads_started":1}
I20260811 12:09:00.155637 5679 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38125
I20260811 12:09:00.156078 5679 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:00.156965 5679 heartbeater.cc:511] Master 127.4.48.254:38125 requested a full tablet report, sending...
I20260811 12:09:00.159610 5342 ts_manager.cc:194] Registered new tserver with Master: 2186a8103a2645ffa79f40224503db26 (127.4.48.193:35513)
I20260811 12:09:00.161346 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:41449
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:00.186766 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:00.505949 5693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:00.506201 5693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:00.506294 5693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:00.513309 5693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:00.513410 5693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:00.513521 5693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:00.522858 5693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:00.524057 5693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:00.525403 5693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:00.538252 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:00.538278 5708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:00.538417 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:00.539139 5693 server_base.cc:1034] running on GCE node
I20260811 12:09:00.539702 5693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:00.540558 5693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:00.541751 5693 hybrid_clock.cc:648] HybridClock initialized: now 1786450140541698 us; error 43 us; skew 500 ppm
I20260811 12:09:00.542044 5693 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:00 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450140, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:00.544952 5693 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:00.546305 5693 webserver.cc:460] Webserver started at http://127.4.48.194:37407/ using document root <none> and password file <none>
I20260811 12:09:00.546960 5693 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:00.547060 5693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:00.547308 5693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:00.550071 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "fcfd1c90349a4d349f8eda389ccce8fd"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "ce7994aa39b101ef7ac3a8abcc3b2cd5"
server_key_iv: "863e455b28bccbb3dea6034c36617ff0"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.550771 5693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "fcfd1c90349a4d349f8eda389ccce8fd"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "ce7994aa39b101ef7ac3a8abcc3b2cd5"
server_key_iv: "863e455b28bccbb3dea6034c36617ff0"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.555948 5693 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:09:00.559943 5720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.561188 5693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:00.561329 5693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "fcfd1c90349a4d349f8eda389ccce8fd"
format_stamp: "Formatted at 2026-08-11 12:09:00 on dist-test-slave-nj21"
server_key: "ce7994aa39b101ef7ac3a8abcc3b2cd5"
server_key_iv: "863e455b28bccbb3dea6034c36617ff0"
server_key_version: "encryptionkey@0"
I20260811 12:09:00.561496 5693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:00.574635 5693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:00.578110 5693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:00.578351 5693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:00.579201 5693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:00.580508 5693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:00.580580 5693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.580685 5693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:00.580736 5693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:00.609542 5693 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:38421
I20260811 12:09:00.609588 5921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:38421 every 8 connection(s)
I20260811 12:09:00.610759 5693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:00.612128 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5693
I20260811 12:09:00.612313 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:00.612778 4291 external_mini_cluster.cc:1428] Setting key e453be80139b2bc550e98281e61106ff
Aug 11 12:09:00 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450140, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:00.631404 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:00.613146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:53787 (local address 127.4.48.254:38125)
0811 12:09:00.613350 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:00.613356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:00.614721 (+ 1365us) server_negotiation.cc:407] Connection header received
0811 12:09:00.616664 (+ 1943us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:00.616670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:00.616752 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:00.616855 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:00.618234 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.619178 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.620825 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.621410 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.621529 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:00.625242 (+ 3713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:00.625249 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:00.625254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:00.625298 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:00.628313 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:00.628996 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:00.629001 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:00.629004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:00.629169 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:00.629643 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:00.629648 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:00.629652 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:00.630030 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:00.630182 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:00.630799 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 12:09:00.631029 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:09:00.632079 5924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:00.613688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38125 (local address 127.4.48.194:53787)
0811 12:09:00.614575 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:00.614623 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:00.616404 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:00.617049 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:00.617056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:00.617531 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:00.618073 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:00.618090 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.619319 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.619323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:00.620696 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:00.620709 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:00.621645 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:00.621652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:00.621927 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:00.622581 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:00.622610 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:00.625000 (+ 2390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:00.628448 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:00.628454 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:00.628465 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:00.628880 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:00.629270 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:00.629273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:00.629275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:00.629513 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:00.630145 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:00.630151 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:00.630451 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 12:09:00.630656 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 12:09:00.631018 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":331,"threads_started":1}
I20260811 12:09:00.633479 5922 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38125
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
I20260811 12:09:00.633774 5922 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:00.634528 5922 heartbeater.cc:511] Master 127.4.48.254:38125 requested a full tablet report, sending...
I20260811 12:09:00.635977 5342 ts_manager.cc:194] Registered new tserver with Master: fcfd1c90349a4d349f8eda389ccce8fd (127.4.48.194:38421)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
I20260811 12:09:00.636909 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:53787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:00.657122 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:00.977217 5928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:00.977507 5928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:00.977666 5928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:00.984776 5928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:00.984896 5928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:00.985069 5928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:00.994462 5928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38125
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:00.995771 5928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:00.997259 5928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:01.009827 5937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:01.009802 5938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:01.009863 5941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:01.010720 5928 server_base.cc:1034] running on GCE node
I20260811 12:09:01.011355 5928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:01.012287 5928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:01.013506 5928 hybrid_clock.cc:648] HybridClock initialized: now 1786450141013452 us; error 38 us; skew 500 ppm
I20260811 12:09:01.013864 5928 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:01.016965 5928 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:01.018405 5928 webserver.cc:460] Webserver started at http://127.4.48.195:45339/ using document root <none> and password file <none>
I20260811 12:09:01.019114 5928 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:01.019250 5928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:01.019565 5928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:01.022683 5928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "79259e380b0f41dcaef9d940c4214903"
format_stamp: "Formatted at 2026-08-11 12:09:01 on dist-test-slave-nj21"
server_key: "83c5fcdb2fdc354d3d6068b566dd76ea"
server_key_iv: "40083d0bed1dd3d0d7fc0815259aee54"
server_key_version: "encryptionkey@0"
I20260811 12:09:01.023619 5928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "79259e380b0f41dcaef9d940c4214903"
format_stamp: "Formatted at 2026-08-11 12:09:01 on dist-test-slave-nj21"
server_key: "83c5fcdb2fdc354d3d6068b566dd76ea"
server_key_iv: "40083d0bed1dd3d0d7fc0815259aee54"
server_key_version: "encryptionkey@0"
I20260811 12:09:01.029036 5928 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:09:01.032390 5952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:01.033591 5928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:01.033735 5928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "79259e380b0f41dcaef9d940c4214903"
format_stamp: "Formatted at 2026-08-11 12:09:01 on dist-test-slave-nj21"
server_key: "83c5fcdb2fdc354d3d6068b566dd76ea"
server_key_iv: "40083d0bed1dd3d0d7fc0815259aee54"
server_key_version: "encryptionkey@0"
I20260811 12:09:01.033901 5928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:01.048182 5928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:01.051638 5928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:01.051914 5928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:01.052794 5928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:01.054028 5928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:01.054107 5928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:01.054234 5928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:01.054292 5928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:01.088879 5928 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:45365
I20260811 12:09:01.088958 6145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:45365 every 8 connection(s)
I20260811 12:09:01.090229 5928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:01.092594 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5928
I20260811 12:09:01.092814 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:01.093290 4291 external_mini_cluster.cc:1428] Setting key a9efd6f105f61f67174a429f4cf75cc0
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:01.110984 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.092928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35763 (local address 127.4.48.254:38125)
0811 12:09:01.093144 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.093151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.094560 (+ 1409us) server_negotiation.cc:407] Connection header received
0811 12:09:01.096993 (+ 2433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.096997 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.097101 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.097180 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.098650 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.099464 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.101046 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.101473 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.101563 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.104986 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.104991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.104994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.105027 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.107884 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.108578 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.108583 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.108587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.108769 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.109212 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.109218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.109222 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.109535 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.109677 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.110382 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.110597 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:09:01.111982 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.093475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38125 (local address 127.4.48.195:35763)
0811 12:09:01.094418 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.094463 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.096753 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.097356 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.097373 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.097910 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:01.098502 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.098519 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.099579 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.099583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.100917 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.100929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.101641 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.101645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.101842 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.102472 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:01.102493 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:01.104787 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:01.108019 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:01.108026 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:01.108038 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:01.108454 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:01.108886 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:01.108889 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:01.108892 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:01.109099 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:01.109671 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:01.109679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:01.110004 (+ 325us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.110228 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.110741 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":394,"threads_started":1}
I20260811 12:09:01.113283 6148 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38125
I20260811 12:09:01.113561 6148 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:01.114279 6148 heartbeater.cc:511] Master 127.4.48.254:38125 requested a full tablet report, sending...
I20260811 12:09:01.115752 5342 ts_manager.cc:194] Registered new tserver with Master: 79259e380b0f41dcaef9d940c4214903 (127.4.48.195:45365)
I20260811 12:09:01.116701 5342 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:35763
I20260811 12:09:01.120682 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:01.142089 5682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.128100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33610 (local address 127.4.48.254:38125)
0811 12:09:01.128270 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.128274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.128904 (+ 630us) server_negotiation.cc:407] Connection header received
0811 12:09:01.129069 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.129073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.129132 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.129210 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.131126 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.131909 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.133215 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.133648 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.133764 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.136256 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.136261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.136264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.136295 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.138861 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.139759 (+ 898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.139764 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.139767 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.139953 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.140384 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.140390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.140394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.140798 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.141070 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.141464 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.141686 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:09:01.150785 5342 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33610:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:01.159613 5342 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:01.164906 5679 heartbeater.cc:503] Master 127.4.48.254:38125 was elected leader, sending a full tablet report...
I20260811 12:09:01.193289 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.176159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60646 (local address 127.4.48.194:38421)
0811 12:09:01.184293 (+ 8134us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.184315 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.184347 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:09:01.184439 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.184445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.184658 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.184859 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:01.185680 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.187453 (+ 1773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.190403 (+ 2950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.191841 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.191987 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.192188 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.192698 (+ 510us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.192886 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7953,"thread_start_us":7833,"threads_started":1}
I20260811 12:09:01.194115 6166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.176019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:38421 (local address 127.0.0.1:60646)
0811 12:09:01.177103 (+ 1084us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.177167 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.177373 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.184915 (+ 7542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.184931 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.184964 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:01.185426 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.185440 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.187582 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.187587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.190262 (+ 2675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.190273 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.192136 (+ 1863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.192149 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.192391 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.192468 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.193494 (+ 1026us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.193700 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":689,"threads_started":1}
I20260811 12:09:01.195755 6175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.177337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34942 (local address 127.4.48.195:45365)
0811 12:09:01.187566 (+ 10229us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.187575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.187617 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:09:01.187730 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.187738 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.187966 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.188167 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:01.189235 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.190766 (+ 1531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.193437 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.194660 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.194792 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.195032 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.195237 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.195419 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10016,"thread_start_us":9828,"threads_started":1}
I20260811 12:09:01.195806 6169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.176719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:45365 (local address 127.0.0.1:34942)
0811 12:09:01.177972 (+ 1253us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.178001 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.178112 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.188159 (+ 10047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.188164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.188189 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:01.189004 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.189021 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.190900 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.190905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.193242 (+ 2337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.193256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.194871 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.194878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.195069 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.195101 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.195180 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.195316 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1103,"mutex_wait_us":48,"thread_start_us":964,"threads_started":1}
I20260811 12:09:01.197250 6178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.177505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35513 (local address 127.0.0.1:51168)
0811 12:09:01.186517 (+ 9012us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.186549 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.186681 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.189143 (+ 2462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.189148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.189173 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:01.189706 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.189719 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.191426 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.191432 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.193755 (+ 2323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.193768 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.196283 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.196290 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.196480 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.196513 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.196727 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.196864 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8872,"mutex_wait_us":36,"thread_start_us":9254,"threads_started":1}
I20260811 12:09:01.198242 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.178352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.4.48.193:35513)
0811 12:09:01.188300 (+ 9948us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.188309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.188335 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:09:01.188425 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.188432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.188659 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.188856 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:01.189890 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.191285 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.193953 (+ 2668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.195749 (+ 1796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.195887 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.196131 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.197575 (+ 1444us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.197785 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9750,"thread_start_us":9550,"threads_started":1}
I20260811 12:09:01.199280 5803 tablet_service.cc:1467] Processing CreateTablet for tablet fc1e975e96724924b4686d14bc13d00c (DEFAULT_TABLE table=test-table [id=52d71fd4070243d5867bd8512f44d5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:01.199412 6037 tablet_service.cc:1467] Processing CreateTablet for tablet fc1e975e96724924b4686d14bc13d00c (DEFAULT_TABLE table=test-table [id=52d71fd4070243d5867bd8512f44d5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:01.200807 5803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc1e975e96724924b4686d14bc13d00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:01.200991 6037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc1e975e96724924b4686d14bc13d00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:01.201930 5559 tablet_service.cc:1467] Processing CreateTablet for tablet fc1e975e96724924b4686d14bc13d00c (DEFAULT_TABLE table=test-table [id=52d71fd4070243d5867bd8512f44d5ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:01.203320 5559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc1e975e96724924b4686d14bc13d00c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:01.213779 6180 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Bootstrap starting.
I20260811 12:09:01.217101 6181 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Bootstrap starting.
I20260811 12:09:01.217669 6180 tablet_bootstrap.cc:654] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:01.218575 6183 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Bootstrap starting.
I20260811 12:09:01.219275 6180 log.cc:824] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:01.221364 6181 tablet_bootstrap.cc:654] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:01.222150 6183 tablet_bootstrap.cc:654] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:01.223003 6181 log.cc:824] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:01.223340 6183 log.cc:824] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:01.226434 6183 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: No bootstrap required, opened a new log
I20260811 12:09:01.226577 6181 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: No bootstrap required, opened a new log
I20260811 12:09:01.226574 6180 tablet_bootstrap.cc:492] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: No bootstrap required, opened a new log
I20260811 12:09:01.226706 6183 ts_tablet_manager.cc:1397] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260811 12:09:01.226863 6181 ts_tablet_manager.cc:1397] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260811 12:09:01.226863 6180 ts_tablet_manager.cc:1397] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Time spent bootstrapping tablet: real 0.013s user 0.011s sys 0.000s
I20260811 12:09:01.234498 6183 raft_consensus.cc:348] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.234858 6183 raft_consensus.cc:374] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:01.234988 6183 raft_consensus.cc:729] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2186a8103a2645ffa79f40224503db26, State: Initialized, Role: FOLLOWER
I20260811 12:09:01.235594 6183 consensus_queue.cc:260] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.237676 6183 ts_tablet_manager.cc:1428] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Time spent starting tablet: real 0.011s user 0.007s sys 0.005s
I20260811 12:09:01.238293 6181 raft_consensus.cc:348] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.238747 6181 raft_consensus.cc:374] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:01.238910 6181 raft_consensus.cc:729] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79259e380b0f41dcaef9d940c4214903, State: Initialized, Role: FOLLOWER
I20260811 12:09:01.239429 6180 raft_consensus.cc:348] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.239542 6181 consensus_queue.cc:260] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.239944 6180 raft_consensus.cc:374] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:01.240087 6180 raft_consensus.cc:729] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcfd1c90349a4d349f8eda389ccce8fd, State: Initialized, Role: FOLLOWER
I20260811 12:09:01.240857 6180 consensus_queue.cc:260] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.246853 6148 heartbeater.cc:503] Master 127.4.48.254:38125 was elected leader, sending a full tablet report...
I20260811 12:09:01.247467 5922 heartbeater.cc:503] Master 127.4.48.254:38125 was elected leader, sending a full tablet report...
I20260811 12:09:01.247918 6181 ts_tablet_manager.cc:1428] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Time spent starting tablet: real 0.021s user 0.018s sys 0.004s
I20260811 12:09:01.247977 6180 ts_tablet_manager.cc:1428] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Time spent starting tablet: real 0.021s user 0.013s sys 0.008s
W20260811 12:09:01.342114 6149 tablet.cc:2369] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:09:01.362759 5923 tablet.cc:2369] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:01.368077 6189 raft_consensus.cc:482] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:01.368372 6189 raft_consensus.cc:504] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.370844 6189 leader_election.cc:288] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fcfd1c90349a4d349f8eda389ccce8fd (127.4.48.194:38421), 2186a8103a2645ffa79f40224503db26 (127.4.48.193:35513)
I20260811 12:09:01.376971 6151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.371200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:38421 (local address 127.4.48.195:35069)
0811 12:09:01.371411 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.371431 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.371559 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.371910 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.371914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.371943 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:01.372185 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.372198 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.373504 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.373510 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.375231 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.375250 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.376433 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.376437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.376577 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.376601 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.376670 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.376758 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":44}
I20260811 12:09:01.377111 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.371126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35069 (local address 127.4.48.194:38421)
0811 12:09:01.371328 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.371333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.371492 (+ 159us) server_negotiation.cc:407] Connection header received
0811 12:09:01.371710 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.371716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.371783 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.371881 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:01.372351 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.373353 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.375401 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.376276 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.376444 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.376519 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.376712 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.376837 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:01.378453 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.371986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:51391 (local address 127.4.48.193:35513)
0811 12:09:01.372197 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.372203 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.372656 (+ 453us) server_negotiation.cc:407] Connection header received
0811 12:09:01.372765 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.372771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.372850 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.372943 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:01.373510 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.374278 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.376397 (+ 2119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.377572 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.377717 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.377769 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.378097 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.378180 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:09:01.378336 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1e975e96724924b4686d14bc13d00c" candidate_uuid: "79259e380b0f41dcaef9d940c4214903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" is_pre_election: true
I20260811 12:09:01.378631 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.371643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35513 (local address 127.4.48.195:51391)
0811 12:09:01.372557 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.372570 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:01.372669 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:01.372958 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:01.372961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:01.372978 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:01.373360 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.373368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.374436 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.374442 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.376242 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:01.376255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.377768 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.377775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:01.377957 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.377988 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:09:01.378084 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:09:01.378205 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":40,"thread_start_us":609,"threads_started":1}
I20260811 12:09:01.379252 5843 raft_consensus.cc:2454] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79259e380b0f41dcaef9d940c4214903 in term 0.
I20260811 12:09:01.379560 5594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1e975e96724924b4686d14bc13d00c" candidate_uuid: "79259e380b0f41dcaef9d940c4214903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2186a8103a2645ffa79f40224503db26" is_pre_election: true
I20260811 12:09:01.380122 5953 leader_election.cc:302] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79259e380b0f41dcaef9d940c4214903, fcfd1c90349a4d349f8eda389ccce8fd; no voters:
I20260811 12:09:01.380223 5594 raft_consensus.cc:2454] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79259e380b0f41dcaef9d940c4214903 in term 0.
I20260811 12:09:01.380573 6189 raft_consensus.cc:2781] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:01.380707 6189 raft_consensus.cc:482] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:01.380797 6189 raft_consensus.cc:3037] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:01.383208 6189 raft_consensus.cc:504] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.383875 6189 leader_election.cc:288] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [CANDIDATE]: Term 1 election: Requested vote from peers fcfd1c90349a4d349f8eda389ccce8fd (127.4.48.194:38421), 2186a8103a2645ffa79f40224503db26 (127.4.48.193:35513)
I20260811 12:09:01.384229 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1e975e96724924b4686d14bc13d00c" candidate_uuid: "79259e380b0f41dcaef9d940c4214903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fcfd1c90349a4d349f8eda389ccce8fd"
I20260811 12:09:01.384387 5594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc1e975e96724924b4686d14bc13d00c" candidate_uuid: "79259e380b0f41dcaef9d940c4214903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2186a8103a2645ffa79f40224503db26"
I20260811 12:09:01.384423 5843 raft_consensus.cc:3037] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:01.384593 5594 raft_consensus.cc:3037] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:09:01.384791 5681 tablet.cc:2369] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:01.387073 5594 raft_consensus.cc:2454] T fc1e975e96724924b4686d14bc13d00c P 2186a8103a2645ffa79f40224503db26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79259e380b0f41dcaef9d940c4214903 in term 1.
I20260811 12:09:01.387127 5843 raft_consensus.cc:2454] T fc1e975e96724924b4686d14bc13d00c P fcfd1c90349a4d349f8eda389ccce8fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79259e380b0f41dcaef9d940c4214903 in term 1.
I20260811 12:09:01.387437 5953 leader_election.cc:302] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2186a8103a2645ffa79f40224503db26, 79259e380b0f41dcaef9d940c4214903; no voters:
I20260811 12:09:01.387734 6189 raft_consensus.cc:2781] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:01.388232 6189 raft_consensus.cc:686] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [term 1 LEADER]: Becoming Leader. State: Replica: 79259e380b0f41dcaef9d940c4214903, State: Running, Role: LEADER
I20260811 12:09:01.388785 6189 consensus_queue.cc:237] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } }
I20260811 12:09:01.394002 5342 catalog_manager.cc:5495] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 reported cstate change: term changed from 0 to 1, leader changed from <none> to 79259e380b0f41dcaef9d940c4214903 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "79259e380b0f41dcaef9d940c4214903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fcfd1c90349a4d349f8eda389ccce8fd" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 38421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2186a8103a2645ffa79f40224503db26" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35513 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79259e380b0f41dcaef9d940c4214903" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 45365 } health_report { overall_health: HEALTHY } } }
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450138, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:01.433960 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.418244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.4.48.193:35513)
0811 12:09:01.418448 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.418454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.419040 (+ 586us) server_negotiation.cc:407] Connection header received
0811 12:09:01.419176 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.419180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.419277 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.419366 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.421173 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.421957 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.423340 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.423845 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.423938 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.428174 (+ 4236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.428188 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.428204 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.428235 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.431222 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.431858 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.431864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.431867 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.432004 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.432374 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.432379 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.432381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.432852 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.433112 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.433449 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.433597 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:01.484035 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.469164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.4.48.193:35513)
0811 12:09:01.469393 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.469399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.469629 (+ 230us) server_negotiation.cc:407] Connection header received
0811 12:09:01.469779 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.469783 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.469850 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.469932 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.471381 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.472256 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.473535 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.473942 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.474032 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.478065 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.478071 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.478077 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.478115 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.481607 (+ 3492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.482188 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.482194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.482198 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.482365 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.482713 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.482718 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.482721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.483058 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.483318 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.483460 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.483698 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:01.530918 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.516917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51198 (local address 127.4.48.193:35513)
0811 12:09:01.517124 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.517131 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.517394 (+ 263us) server_negotiation.cc:407] Connection header received
0811 12:09:01.517540 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.517544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.517608 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.517679 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.519115 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.519867 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.521090 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.521565 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.521654 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.525630 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.525635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.525638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.525678 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.528303 (+ 2625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.528940 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.528944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.528946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.529069 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.529436 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.529453 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.529460 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.529775 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.530047 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.530323 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.530571 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 12:09:01.533260 5526 tablet_service.cc:2908] Rejecting scan request for tablet fc1e975e96724924b4686d14bc13d00c: Uninitialized: clean time has not yet been initialized
I20260811 12:09:01.536769 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5450
I20260811 12:09:01.549305 6179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.537644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51212 (local address 127.4.48.193:35513)
0811 12:09:01.537794 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.537799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.538067 (+ 268us) server_negotiation.cc:407] Connection header received
0811 12:09:01.538212 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.538217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.538303 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.538412 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.539477 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.540172 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.541338 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.541795 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.541924 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.544231 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.544236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.544239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.544267 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.546643 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.547381 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.547386 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.547390 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.547522 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.547899 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.547905 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.547908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.548200 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.548528 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.548784 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.549024 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:09:01.549926 5668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:01.637374 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5450
I20260811 12:09:01.661893 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5693
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:01.676136 6167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.662717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60656 (local address 127.4.48.194:38421)
0811 12:09:01.662879 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.662884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.662902 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:09:01.662960 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.662964 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.663054 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.663137 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.664436 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.665300 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.666548 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.666960 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.667049 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.670789 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.670804 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.670819 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.670860 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.673498 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.674063 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.674069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.674071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.674191 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.674511 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.674514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.674516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.674958 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.675283 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.675558 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.675795 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:09:01.676981 5915 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:01.765998 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5693
I20260811 12:09:01.789155 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5928
I20260811 12:09:01.790303 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.789725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:38421 (local address 127.4.48.195:40703)
0811 12:09:01.789895 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.790096 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.194:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:09:01.790612 5953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.194:38421: connect: Connection refused (error 111)
I20260811 12:09:01.792263 6192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.791875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:38421 (local address 127.4.48.195:48025)
0811 12:09:01.792026 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:01.792127 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.194:38421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 12:09:01.792543 5953 consensus_peers.cc:487] T fc1e975e96724924b4686d14bc13d00c P 79259e380b0f41dcaef9d940c4214903 -> Peer fcfd1c90349a4d349f8eda389ccce8fd (127.4.48.194:38421): Couldn't send request to peer fcfd1c90349a4d349f8eda389ccce8fd. Status: Network error: Client connection negotiation failed: client connection to 127.4.48.194:38421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
Aug 11 12:09:01 dist-test-slave-nj21 krb5kdc[5268](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:01.803984 6227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:01.789886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34946 (local address 127.4.48.195:45365)
0811 12:09:01.790318 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:01.790322 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:01.790338 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:01.790386 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:01.790389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:01.790481 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:01.790592 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:01.792053 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.792848 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.794046 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:01.794429 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:01.794514 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:01.798729 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:01.798742 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:01.798758 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:01.798793 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:01.801607 (+ 2814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.802182 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.802189 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.802201 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.802341 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:01.802629 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:01.802632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:01.802634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:01.802989 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:01.803230 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:01.803408 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 12:09:01.803577 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":224,"threads_started":1}
I20260811 12:09:01.804615 6136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:01.892560 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5928
I20260811 12:09:01.917511 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5290
I20260811 12:09:01.918725 5403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:02.024868 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5290
2026-08-11T12:09:02Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4407 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:02 dist-test-slave-nj21 krb5kdc[6238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:02 dist-test-slave-nj21 krb5kdc[6238](info): set up 2 sockets
Aug 11 12:09:02 dist-test-slave-nj21 krb5kdc[6238](info): commencing operation
krb5kdc: starting...
W20260811 12:09:03.269726 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:03 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:03.450122 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38047
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38047
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:03.778836 6263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:03.779103 6263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:03.779229 6263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:03.786247 6263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:03.786356 6263 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:09:03.786422 6263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:03.786506 6263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:03.786581 6263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:03.786672 6263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:03.796661 6263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38047
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38047
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:03.797963 6263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:03.799345 6263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:03.810596 6273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:03.810762 6280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:03.810688 6275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:03.811702 6263 server_base.cc:1034] running on GCE node
I20260811 12:09:03.812456 6263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:03.813781 6263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:03.814992 6263 hybrid_clock.cc:648] HybridClock initialized: now 1786450143814924 us; error 49 us; skew 500 ppm
I20260811 12:09:03.815310 6263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:03 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450143, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:03.820906 6263 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:03.822301 6263 webserver.cc:460] Webserver started at http://127.4.48.254:46321/ using document root <none> and password file <none>
I20260811 12:09:03.822981 6263 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:03.823081 6263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:03.823400 6263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:03.826385 6263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "6fbe4f44876d471d8cae505e2524354b"
format_stamp: "Formatted at 2026-08-11 12:09:03 on dist-test-slave-nj21"
server_key: "e368f2921f5044378a4c61e61ad82421"
server_key_iv: "134c952fd2de20b4afaf11c48f5ec7fc"
server_key_version: "encryptionkey@0"
I20260811 12:09:03.827157 6263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "6fbe4f44876d471d8cae505e2524354b"
format_stamp: "Formatted at 2026-08-11 12:09:03 on dist-test-slave-nj21"
server_key: "e368f2921f5044378a4c61e61ad82421"
server_key_iv: "134c952fd2de20b4afaf11c48f5ec7fc"
server_key_version: "encryptionkey@0"
I20260811 12:09:03.833035 6263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:09:03.836375 6288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:03.837615 6263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:03.837781 6263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "6fbe4f44876d471d8cae505e2524354b"
format_stamp: "Formatted at 2026-08-11 12:09:03 on dist-test-slave-nj21"
server_key: "e368f2921f5044378a4c61e61ad82421"
server_key_iv: "134c952fd2de20b4afaf11c48f5ec7fc"
server_key_version: "encryptionkey@0"
I20260811 12:09:03.837981 6263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:03.869561 6263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:03.873365 6263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:03.873644 6263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:03.894423 6263 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:38047
I20260811 12:09:03.894397 6398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:38047 every 8 connection(s)
I20260811 12:09:03.895721 6263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:03.899313 6403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:03.899886 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6263
I20260811 12:09:03.900106 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:03.900954 4291 external_mini_cluster.cc:1428] Setting key c942d8b8357a6e1da0664bcc30f20e0b
I20260811 12:09:03.912863 6403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Bootstrap starting.
Aug 11 12:09:03 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:03.916688 6403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:03.917760 6403 log.cc:824] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:03.920935 6403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: No bootstrap required, opened a new log
I20260811 12:09:03.924381 6407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:03.902984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.4.48.254:38047)
0811 12:09:03.903819 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:03.903831 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:03.903877 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:09:03.905612 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:03.905634 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:03.906016 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:03.906322 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:03.907995 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:03.909279 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:03.910968 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:03.911589 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:03.911748 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:03.916572 (+ 4824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:03.916579 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:03.916597 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:03.916633 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:03.920148 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:03.920768 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:03.920776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:03.920783 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:03.920998 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:03.921363 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:03.921368 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:03.921371 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:03.922057 (+ 686us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:03.922361 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:03.922634 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 12:09:03.923074 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":388,"threads_started":1}
I20260811 12:09:03.929201 6403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } }
I20260811 12:09:03.929510 6403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:03.929590 6403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fbe4f44876d471d8cae505e2524354b, State: Initialized, Role: FOLLOWER
I20260811 12:09:03.930076 6403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } }
I20260811 12:09:03.930285 6403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:03.930390 6403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:03.930541 6403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:03.932646 6403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } }
I20260811 12:09:03.933099 6403 leader_election.cc:302] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fbe4f44876d471d8cae505e2524354b; no voters:
I20260811 12:09:03.933605 6403 leader_election.cc:288] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:03.933722 6409 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:03.934325 6409 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [term 1 LEADER]: Becoming Leader. State: Replica: 6fbe4f44876d471d8cae505e2524354b, State: Running, Role: LEADER
I20260811 12:09:03.934947 6409 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } }
I20260811 12:09:03.935667 6403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:03.937857 6413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fbe4f44876d471d8cae505e2524354b. Latest consensus state: current_term: 1 leader_uuid: "6fbe4f44876d471d8cae505e2524354b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } } }
I20260811 12:09:03.938292 6413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:03.938413 6412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fbe4f44876d471d8cae505e2524354b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fbe4f44876d471d8cae505e2524354b" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38047 } } }
I20260811 12:09:03.938741 6412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:03.940434 6424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:03.945629 6424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:03.953155 6424 catalog_manager.cc:1347] Generated new cluster ID: f7c1340d7e49483d8b1c8e1d14a90627
I20260811 12:09:03.953274 6424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:03.975962 6424 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:03.977169 6424 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:03.984118 6424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Generated new TSK 0
I20260811 12:09:03.984809 6424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:04.038676 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38047
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:04.379902 6443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:04.380153 6443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:04.380256 6443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:04.387068 6443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:04.387166 6443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:04.387252 6443 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:04.387329 6443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:04.396502 6443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:04.397776 6443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:04.399026 6443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:04.411429 6449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:04.411482 6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:04.411453 6450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:04.412252 6443 server_base.cc:1034] running on GCE node
I20260811 12:09:04.412797 6443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:04.413606 6443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:04.414812 6443 hybrid_clock.cc:648] HybridClock initialized: now 1786450144414756 us; error 31 us; skew 500 ppm
I20260811 12:09:04.415066 6443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:04 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450144, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:04.418203 6443 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:04.419579 6443 webserver.cc:460] Webserver started at http://127.4.48.193:35355/ using document root <none> and password file <none>
I20260811 12:09:04.420239 6443 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:04.420331 6443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:04.420648 6443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:04.423424 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "a9f7cc78e085f306d03873f8f7362d10"
server_key_iv: "0ebfbd47d9e2036a9821b7f28da5f6b9"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.424204 6443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "a9f7cc78e085f306d03873f8f7362d10"
server_key_iv: "0ebfbd47d9e2036a9821b7f28da5f6b9"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.428896 6443 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 12:09:04.432065 6464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:04.433229 6443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260811 12:09:04.433369 6443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "a9f7cc78e085f306d03873f8f7362d10"
server_key_iv: "0ebfbd47d9e2036a9821b7f28da5f6b9"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.433552 6443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:04.449064 6443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:04.452531 6443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:04.452756 6443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:04.455323 6443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:04.455431 6443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:04.455580 6443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:04.455649 6443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:04 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450144, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:04.477710 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.455932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56599 (local address 127.4.48.254:38047)
0811 12:09:04.456783 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:04.456790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:04.457683 (+ 893us) server_negotiation.cc:407] Connection header received
0811 12:09:04.459877 (+ 2194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:04.459884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:04.459982 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:04.460108 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:04.462065 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.463211 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.465548 (+ 2337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.466120 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.466235 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.470840 (+ 4605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:04.470855 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:04.470860 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:04.470905 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:04.474064 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.475081 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.475087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.475091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.475222 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.475764 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.475770 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.475773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.476179 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:04.476338 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:04.477046 (+ 708us) server_negotiation.cc:299] Negotiation successful
0811 12:09:04.477302 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":620,"threads_started":1}
I20260811 12:09:04.478411 6479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.456643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.193:56599)
0811 12:09:04.457500 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:04.457555 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:04.459530 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:04.460413 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:04.460424 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:04.461092 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:04.461866 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.461895 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.463392 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.463400 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.465408 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.465420 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.466375 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.466383 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.466634 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.467341 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:04.467377 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:04.470585 (+ 3208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:04.474202 (+ 3617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.474213 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.474233 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.474914 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.475331 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.475336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.475340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.475590 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.476327 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:04.476336 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:04.476687 (+ 351us) client_negotiation.cc:769] Sending connection context
0811 12:09:04.476906 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 12:09:04.477263 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":352,"threads_started":1}
I20260811 12:09:04.496732 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.484290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37771 (local address 127.4.48.254:38047)
0811 12:09:04.484441 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:04.484446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:04.485099 (+ 653us) server_negotiation.cc:407] Connection header received
0811 12:09:04.485242 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:04.485247 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:04.485314 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:04.485392 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:04.486309 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.487018 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.488564 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.489115 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.489225 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.491314 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:04.491320 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:04.491324 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:04.491355 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:04.494037 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.494751 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.494756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.494760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.494929 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.495309 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.495314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.495318 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.495629 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:04.495807 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:04.496189 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 12:09:04.496397 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:09:04.496819 6587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.484403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.193:37771)
0811 12:09:04.485005 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:04.485020 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:04.485140 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:04.485428 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:04.485432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:04.485871 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:04.486177 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.486185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.487137 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.487141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.488414 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.488436 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.489305 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.489311 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.489492 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.490160 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:04.490180 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:04.491141 (+ 961us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:04.494159 (+ 3018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.494164 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.494171 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.494586 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.495022 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.495027 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.495031 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.495155 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.495770 (+ 615us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:04.495775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:04.495956 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 12:09:04.496073 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:09:04.496388 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":339,"threads_started":1}
I20260811 12:09:04.506781 6443 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:36859
I20260811 12:09:04.506831 6656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:36859 every 8 connection(s)
I20260811 12:09:04.507934 6443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:04.508916 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6443
I20260811 12:09:04.509133 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:04.509773 4291 external_mini_cluster.cc:1428] Setting key 83dde652caafd92cfa1259d2dd1c073a
I20260811 12:09:04.509862 6657 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38047
I20260811 12:09:04.510087 6657 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:04.510859 6657 heartbeater.cc:511] Master 127.4.48.254:38047 requested a full tablet report, sending...
I20260811 12:09:04.512698 6318 ts_manager.cc:194] Registered new tserver with Master: b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193:36859)
I20260811 12:09:04.514075 6318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:56599
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:04.550448 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38047
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:04.864646 6661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:04.864914 6661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:04.865053 6661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:04.871958 6661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:04.872069 6661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:04.872188 6661 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:04.872295 6661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:04.882077 6661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:04.883339 6661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:04.884727 6661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:04.897964 6671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:04.898126 6676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:04.898399 6670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:04.899297 6661 server_base.cc:1034] running on GCE node
I20260811 12:09:04.900007 6661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:04.901012 6661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:04.902267 6661 hybrid_clock.cc:648] HybridClock initialized: now 1786450144902194 us; error 31 us; skew 500 ppm
I20260811 12:09:04.902609 6661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:04 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450144, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:04.906068 6661 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:04.911783 6661 webserver.cc:460] Webserver started at http://127.4.48.194:44939/ using document root <none> and password file <none>
I20260811 12:09:04.912438 6661 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:04.912533 6661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:04.912878 6661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:04.915642 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "2314794b2aea416c966d32795c02899e"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "f509ab4cb1ca78524eec9b13af61c6f5"
server_key_iv: "46c515cbe422aec4c90793e34f32b811"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.916458 6661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "2314794b2aea416c966d32795c02899e"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "f509ab4cb1ca78524eec9b13af61c6f5"
server_key_iv: "46c515cbe422aec4c90793e34f32b811"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.921283 6661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:04.924557 6683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:04.925724 6661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:04.925859 6661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "2314794b2aea416c966d32795c02899e"
format_stamp: "Formatted at 2026-08-11 12:09:04 on dist-test-slave-nj21"
server_key: "f509ab4cb1ca78524eec9b13af61c6f5"
server_key_iv: "46c515cbe422aec4c90793e34f32b811"
server_key_version: "encryptionkey@0"
I20260811 12:09:04.926088 6661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:04.946036 6661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:04.950023 6661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:04.950260 6661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:04.952754 6661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:04.952854 6661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:04.953001 6661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:04.953068 6661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:04 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450144, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:04.974258 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.953414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:41347 (local address 127.4.48.254:38047)
0811 12:09:04.954582 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:04.954588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:04.955033 (+ 445us) server_negotiation.cc:407] Connection header received
0811 12:09:04.957172 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:04.957179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:04.957304 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:04.957419 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:04.958779 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.959951 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.962853 (+ 2902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.963363 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.963498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.967904 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:04.967912 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:04.967917 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:04.967963 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:04.970738 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.971715 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.971720 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.971724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.971877 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.972417 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.972423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.972426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.972810 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:04.972997 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:04.973654 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 12:09:04.973909 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1032}
I20260811 12:09:04.975584 6703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.954079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.194:41347)
0811 12:09:04.954883 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:04.954925 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:04.956892 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:04.957647 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:04.957655 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:04.958102 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:04.958631 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.958646 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.960154 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.960163 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.962685 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.962705 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.963611 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.963621 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.963955 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.964668 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:04.964704 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:04.967633 (+ 2929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:04.970880 (+ 3247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.970891 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.970907 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.971567 (+ 660us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.972015 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.972019 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.972021 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.972280 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.972957 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:04.972963 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:04.973274 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 12:09:04.973558 (+ 284us) client_negotiation.cc:240] Negotiation successful
0811 12:09:04.974068 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":401,"threads_started":1}
I20260811 12:09:04.995055 6661 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:36803
I20260811 12:09:04.995205 6901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:36803 every 8 connection(s)
I20260811 12:09:04.996040 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.982231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:60539 (local address 127.4.48.254:38047)
0811 12:09:04.982394 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:04.982400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:04.983140 (+ 740us) server_negotiation.cc:407] Connection header received
0811 12:09:04.983284 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:04.983288 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:04.983353 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:04.983443 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:04.984931 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.985839 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.987460 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.988025 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.988150 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.990357 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:04.990372 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:04.990377 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:04.990416 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:04.993431 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.994032 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.994038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.994041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.994196 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:04.994596 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:04.994601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:04.994605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:04.995005 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:04.995156 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:04.995615 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 12:09:04.995822 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:09:04.996307 6836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:04.982292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.194:60539)
0811 12:09:04.983028 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:04.983050 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:04.983185 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:04.983598 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:04.983606 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:04.984241 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:04.984711 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.984725 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.986027 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.986033 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.987314 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:04.987326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:04.988222 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:04.988228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:04.988445 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:04.989120 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:04.989137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:04.990181 (+ 1044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:04.993546 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.993551 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.993556 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.993927 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.994338 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:04.994342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:04.994345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:04.994490 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:04.995156 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:04.995160 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:04.995384 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 12:09:04.995536 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 12:09:04.995877 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":432,"threads_started":1}
I20260811 12:09:04.996692 6661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:04.999184 6903 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38047
I20260811 12:09:04.999398 6903 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:05.000072 6903 heartbeater.cc:511] Master 127.4.48.254:38047 requested a full tablet report, sending...
I20260811 12:09:05.001358 6318 ts_manager.cc:194] Registered new tserver with Master: 2314794b2aea416c966d32795c02899e (127.4.48.194:36803)
I20260811 12:09:05.002135 6318 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:41347
I20260811 12:09:05.005365 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6661
I20260811 12:09:05.005537 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:05.005952 4291 external_mini_cluster.cc:1428] Setting key df2381669be0527864c6b139854becdf
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:05.046346 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:38047
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41981
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:05.361094 6910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:05.361349 6910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:05.361480 6910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:05.368427 6910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:05.368530 6910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:05.368652 6910 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:05.368762 6910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:05.378131 6910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41981
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:38047
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:05.379345 6910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:05.380688 6910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:05.393034 6919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:05.393071 6923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:05.393101 6920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:05.394158 6910 server_base.cc:1034] running on GCE node
I20260811 12:09:05.394711 6910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:05.395576 6910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:05.396772 6910 hybrid_clock.cc:648] HybridClock initialized: now 1786450145396695 us; error 52 us; skew 500 ppm
I20260811 12:09:05.397047 6910 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:05 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:05.399988 6910 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:05.401257 6910 webserver.cc:460] Webserver started at http://127.4.48.195:46087/ using document root <none> and password file <none>
I20260811 12:09:05.401856 6910 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:05.401949 6910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:05.402220 6910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:05.405031 6910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "aefd28047a54456692699f4a4519808f"
format_stamp: "Formatted at 2026-08-11 12:09:05 on dist-test-slave-nj21"
server_key: "40b6bb43366c6340979b6281e071f17d"
server_key_iv: "ba94f6f5aeda22202213eedf820b9ea5"
server_key_version: "encryptionkey@0"
I20260811 12:09:05.405743 6910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "aefd28047a54456692699f4a4519808f"
format_stamp: "Formatted at 2026-08-11 12:09:05 on dist-test-slave-nj21"
server_key: "40b6bb43366c6340979b6281e071f17d"
server_key_iv: "ba94f6f5aeda22202213eedf820b9ea5"
server_key_version: "encryptionkey@0"
I20260811 12:09:05.410706 6910 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 12:09:05.414220 6944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:05.415410 6910 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:09:05.415551 6910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "aefd28047a54456692699f4a4519808f"
format_stamp: "Formatted at 2026-08-11 12:09:05 on dist-test-slave-nj21"
server_key: "40b6bb43366c6340979b6281e071f17d"
server_key_iv: "ba94f6f5aeda22202213eedf820b9ea5"
server_key_version: "encryptionkey@0"
I20260811 12:09:05.415781 6910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:05.428539 6910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:05.432199 6910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:05.432425 6910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:05.434787 6910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:05.434895 6910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:05.435035 6910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:05.435104 6910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:05 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:05.458555 6957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.435985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.195:45943)
0811 12:09:05.436767 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.436808 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.438680 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.439376 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.439388 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.440122 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:05.441106 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.441133 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.442408 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.442413 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.444630 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.444645 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.445521 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.445528 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.445790 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.446484 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:05.446518 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:05.449600 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:05.453380 (+ 3780us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:05.453392 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:05.453406 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:05.453998 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:05.454443 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:05.454449 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:05.454454 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:05.454759 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:05.455336 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:05.455345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:05.455872 (+ 527us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.456224 (+ 352us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.456832 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260811 12:09:05.460044 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.435386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:45943 (local address 127.4.48.254:38047)
0811 12:09:05.435811 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.435817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.436914 (+ 1097us) server_negotiation.cc:407] Connection header received
0811 12:09:05.438944 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.438949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.439036 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.439122 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:05.441277 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.442258 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.444818 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.445296 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.445418 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.449820 (+ 4402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:05.449826 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:05.449831 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:05.449873 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:05.453238 (+ 3365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.454171 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.454177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.454181 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.454316 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.454903 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.454907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.454910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.455200 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:05.455376 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.459414 (+ 4038us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.459686 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266}
I20260811 12:09:05.477182 6910 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:42343
I20260811 12:09:05.477269 7137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:42343 every 8 connection(s)
I20260811 12:09:05.478470 6910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:05.479387 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.465518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:55225 (local address 127.4.48.254:38047)
0811 12:09:05.465661 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.465664 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.466443 (+ 779us) server_negotiation.cc:407] Connection header received
0811 12:09:05.466596 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.466602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.466669 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.466744 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:05.467974 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.469020 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.470586 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.471243 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.471382 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.473835 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:05.473842 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:05.473846 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:05.473884 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:05.476680 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.477489 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.477493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.477495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.477627 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.477966 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.477971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.477975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.478265 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:05.478405 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.478830 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.479040 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:09:05.479530 7082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.465611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.4.48.195:55225)
0811 12:09:05.466338 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.466361 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.466487 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.466790 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.466794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.467361 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:05.467823 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.467841 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.469150 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.469153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.470438 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.470460 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.471640 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.471646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.471878 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.472625 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:05.472649 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:05.473612 (+ 963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:05.476832 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:05.476837 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:05.476841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:05.477352 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:05.477731 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:05.477736 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:05.477740 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:05.477867 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:05.478375 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:05.478380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:05.478564 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.478724 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.479074 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":372,"threads_started":1}
I20260811 12:09:05.480921 7138 heartbeater.cc:347] Connected to a master server at 127.4.48.254:38047
I20260811 12:09:05.481274 7138 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:05.481392 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6910
I20260811 12:09:05.481578 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:05.481989 4291 external_mini_cluster.cc:1428] Setting key 6a9c91691c46496abdb148abca5bdb57
I20260811 12:09:05.482075 7138 heartbeater.cc:511] Master 127.4.48.254:38047 requested a full tablet report, sending...
I20260811 12:09:05.483604 6317 ts_manager.cc:194] Registered new tserver with Master: aefd28047a54456692699f4a4519808f (127.4.48.195:42343)
I20260811 12:09:05.484727 6317 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:45943
I20260811 12:09:05.485414 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:09:05 dist-test-slave-nj21 krb5kdc[6238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
I20260811 12:09:05.516907 6657 heartbeater.cc:503] Master 127.4.48.254:38047 was elected leader, sending a full tablet report...
Aug 11 12:09:05 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:05.532996 6478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.520343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58684 (local address 127.4.48.254:38047)
0811 12:09:05.520492 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.520496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.520864 (+ 368us) server_negotiation.cc:407] Connection header received
0811 12:09:05.521013 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.521017 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.521075 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.521154 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:05.522455 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.523173 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.524447 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.524860 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.524951 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.528551 (+ 3600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:05.528556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:05.528559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:05.528594 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:05.530960 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.531471 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.531475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.531477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.531597 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:05.531921 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:05.531925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:05.531927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:05.532188 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:05.532377 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.532566 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.532726 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:09:05.537003 6317 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:05.544020 6317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:05.566959 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.556142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51360 (local address 127.4.48.194:36803)
0811 12:09:05.557176 (+ 1034us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.557184 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.557777 (+ 593us) server_negotiation.cc:407] Connection header received
0811 12:09:05.558068 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.558077 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.558347 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.558525 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:05.560161 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.561644 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.564178 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.565530 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.565649 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.565855 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.566459 (+ 604us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.566629 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":877,"thread_start_us":753,"threads_started":1}
I20260811 12:09:05.567310 7171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.555878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36803 (local address 127.0.0.1:51360)
0811 12:09:05.557664 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.557683 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.557885 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.558812 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.558821 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.558852 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:05.559284 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.559295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.561790 (+ 2495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.561794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.563997 (+ 2203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.564020 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.565725 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.565731 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.565924 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.565964 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.566086 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.566269 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1648,"thread_start_us":4102,"threads_started":1}
I20260811 12:09:05.567786 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.555329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36859 (local address 127.0.0.1:45928)
0811 12:09:05.556394 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.556435 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.556614 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.561384 (+ 4770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.561389 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.561413 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:05.561771 (+ 358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.561783 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.563629 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.563635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.565248 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.565260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.566951 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.566958 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.567134 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.567169 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.567229 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.567364 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":25,"thread_start_us":1816,"threads_started":1}
I20260811 12:09:05.568338 7167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.556530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42343 (local address 127.0.0.1:60984)
0811 12:09:05.560234 (+ 3704us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.560250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.560357 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.560984 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.560989 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.561015 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:05.561342 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.561352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.563298 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.563303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.565484 (+ 2181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.565500 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.567519 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.567525 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.567702 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.567740 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.567828 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.567985 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3596,"mutex_wait_us":52,"thread_start_us":1206,"threads_started":1}
I20260811 12:09:05.568617 7170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.556193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60984 (local address 127.4.48.195:42343)
0811 12:09:05.560029 (+ 3836us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.560037 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.560471 (+ 434us) server_negotiation.cc:407] Connection header received
0811 12:09:05.560573 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.560580 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.560792 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.560988 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:05.561565 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.563151 (+ 1586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.565642 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.567040 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.567152 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.567376 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.568146 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.568310 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3631,"thread_start_us":3410,"threads_started":1}
I20260811 12:09:05.569165 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.555514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45928 (local address 127.4.48.193:36859)
0811 12:09:05.560879 (+ 5365us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.560889 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.560913 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:09:05.561003 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.561010 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.561213 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.561384 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:05.561980 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.563492 (+ 1512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.565410 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.566726 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.567612 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.568641 (+ 1029us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.568708 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.568884 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5212,"thread_start_us":5090,"threads_started":1}
I20260811 12:09:05.571310 6785 tablet_service.cc:1467] Processing CreateTablet for tablet 3f431aa44e20475dba596e2e05653fef (DEFAULT_TABLE table=test-table [id=af4e46d4430b4650aafc209efbe73191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.572567 6785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f431aa44e20475dba596e2e05653fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.572206 7025 tablet_service.cc:1467] Processing CreateTablet for tablet 3f431aa44e20475dba596e2e05653fef (DEFAULT_TABLE table=test-table [id=af4e46d4430b4650aafc209efbe73191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.572494 6546 tablet_service.cc:1467] Processing CreateTablet for tablet 3f431aa44e20475dba596e2e05653fef (DEFAULT_TABLE table=test-table [id=af4e46d4430b4650aafc209efbe73191]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.573722 7025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f431aa44e20475dba596e2e05653fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.573720 6546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f431aa44e20475dba596e2e05653fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.585048 7172 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Bootstrap starting.
I20260811 12:09:05.588959 7172 tablet_bootstrap.cc:654] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.590222 7172 log.cc:824] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:05.592514 7175 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Bootstrap starting.
I20260811 12:09:05.593549 7172 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: No bootstrap required, opened a new log
I20260811 12:09:05.591584 7174 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Bootstrap starting.
I20260811 12:09:05.593858 7172 ts_tablet_manager.cc:1397] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:09:05.596529 7175 tablet_bootstrap.cc:654] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.596555 7174 tablet_bootstrap.cc:654] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.597991 7174 log.cc:824] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:05.598160 7175 log.cc:824] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:05.601280 7174 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: No bootstrap required, opened a new log
I20260811 12:09:05.601289 7175 tablet_bootstrap.cc:492] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: No bootstrap required, opened a new log
I20260811 12:09:05.601579 7175 ts_tablet_manager.cc:1397] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:09:05.601596 7174 ts_tablet_manager.cc:1397] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 12:09:05.604727 7172 raft_consensus.cc:348] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.605242 7172 raft_consensus.cc:374] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.605360 7172 raft_consensus.cc:729] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2314794b2aea416c966d32795c02899e, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.606031 7172 consensus_queue.cc:260] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.607312 6903 heartbeater.cc:503] Master 127.4.48.254:38047 was elected leader, sending a full tablet report...
I20260811 12:09:05.608001 7172 ts_tablet_manager.cc:1428] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260811 12:09:05.612218 7175 raft_consensus.cc:348] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.612686 7175 raft_consensus.cc:374] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.612836 7175 raft_consensus.cc:729] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefd28047a54456692699f4a4519808f, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.613507 7175 consensus_queue.cc:260] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.613694 7174 raft_consensus.cc:348] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.614161 7174 raft_consensus.cc:374] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.614272 7174 raft_consensus.cc:729] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b7aec192fc4d959a5113bfd2f8d5fd, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.614840 7138 heartbeater.cc:503] Master 127.4.48.254:38047 was elected leader, sending a full tablet report...
I20260811 12:09:05.614881 7174 consensus_queue.cc:260] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.615465 7175 ts_tablet_manager.cc:1428] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 12:09:05.616645 7174 ts_tablet_manager.cc:1428] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 12:09:05.664321 7182 raft_consensus.cc:482] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:05.664666 7182 raft_consensus.cc:504] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.666400 7182 leader_election.cc:288] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aefd28047a54456692699f4a4519808f (127.4.48.195:42343), 2314794b2aea416c966d32795c02899e (127.4.48.194:36803)
I20260811 12:09:05.675953 7169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.666948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:50699 (local address 127.4.48.194:36803)
0811 12:09:05.667164 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.667170 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.667810 (+ 640us) server_negotiation.cc:407] Connection header received
0811 12:09:05.668022 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.668026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.668104 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.668208 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:05.669169 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.670365 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.673399 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.674727 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.674876 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.674952 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.675526 (+ 574us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.675651 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:09:05.676067 7170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.666870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:59791 (local address 127.4.48.195:42343)
0811 12:09:05.667085 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.667092 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.667854 (+ 762us) server_negotiation.cc:407] Connection header received
0811 12:09:05.668021 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.668026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.668104 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.668198 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:05.669169 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.670337 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.673399 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.674705 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.674880 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.674952 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.675432 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.675652 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:09:05.676149 7185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.666792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42343 (local address 127.4.48.193:59791)
0811 12:09:05.667753 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.667767 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.667906 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.668222 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.668227 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.668274 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:05.669024 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.669032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.670470 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.670474 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.673238 (+ 2764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.673254 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.675018 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.675026 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.675243 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.675285 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.675385 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.675516 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"mutex_wait_us":134,"spinlock_wait_cycles":51968,"thread_start_us":633,"threads_started":1}
I20260811 12:09:05.676733 7184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.666668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36803 (local address 127.4.48.193:50699)
0811 12:09:05.667661 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:05.667711 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:05.667884 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:05.668226 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:05.668230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:05.668272 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:05.669017 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.669031 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.670464 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.670469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.673249 (+ 2780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:05.673259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.675134 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.675139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:05.675334 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.675375 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:09:05.676136 (+ 761us) client_negotiation.cc:240] Negotiation successful
0811 12:09:05.676261 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"mutex_wait_us":134,"spinlock_wait_cycles":17536,"thread_start_us":494,"threads_started":1}
I20260811 12:09:05.677930 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f431aa44e20475dba596e2e05653fef" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f" is_pre_election: true
I20260811 12:09:05.678292 6824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f431aa44e20475dba596e2e05653fef" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2314794b2aea416c966d32795c02899e" is_pre_election: true
I20260811 12:09:05.678676 7058 raft_consensus.cc:2454] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 0.
I20260811 12:09:05.678884 6824 raft_consensus.cc:2454] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 0.
I20260811 12:09:05.679493 6468 leader_election.cc:302] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aefd28047a54456692699f4a4519808f, b6b7aec192fc4d959a5113bfd2f8d5fd; no voters:
I20260811 12:09:05.680294 7182 raft_consensus.cc:2781] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:05.680543 7182 raft_consensus.cc:482] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:05.680671 7182 raft_consensus.cc:3037] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:05.683267 7182 raft_consensus.cc:504] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.683961 7182 leader_election.cc:288] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 election: Requested vote from peers aefd28047a54456692699f4a4519808f (127.4.48.195:42343), 2314794b2aea416c966d32795c02899e (127.4.48.194:36803)
I20260811 12:09:05.684303 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f431aa44e20475dba596e2e05653fef" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f"
I20260811 12:09:05.684437 6824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f431aa44e20475dba596e2e05653fef" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2314794b2aea416c966d32795c02899e"
I20260811 12:09:05.684489 7058 raft_consensus.cc:3037] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:05.684634 6824 raft_consensus.cc:3037] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:05.686878 7058 raft_consensus.cc:2454] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 1.
I20260811 12:09:05.687059 6824 raft_consensus.cc:2454] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 1.
I20260811 12:09:05.687238 6468 leader_election.cc:302] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aefd28047a54456692699f4a4519808f, b6b7aec192fc4d959a5113bfd2f8d5fd; no voters:
I20260811 12:09:05.687502 7182 raft_consensus.cc:2781] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:05.688019 7182 raft_consensus.cc:686] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 LEADER]: Becoming Leader. State: Replica: b6b7aec192fc4d959a5113bfd2f8d5fd, State: Running, Role: LEADER
I20260811 12:09:05.688486 7182 consensus_queue.cc:237] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } }
I20260811 12:09:05.693122 6318 catalog_manager.cc:5495] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd reported cstate change: term changed from 0 to 1, leader changed from <none> to b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193). New cstate: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } health_report { overall_health: UNKNOWN } } }
W20260811 12:09:05.731112 7141 tablet.cc:2369] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:05.733969 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:05.728332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45930 (local address 127.4.48.193:36859)
0811 12:09:05.728595 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:05.728601 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:05.728623 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:09:05.728918 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:05.728922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:05.729018 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:05.729149 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:05.729493 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.730227 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.731979 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:05.732423 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:05.732526 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:05.732939 (+ 413us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:05.733465 (+ 526us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:05.733582 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:05.733632 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 12:09:05.733697 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 12:09:05.743139 7058 raft_consensus.cc:1261] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Refusing update from remote peer b6b7aec192fc4d959a5113bfd2f8d5fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:05.744194 7186 consensus_queue.cc:1035] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:05.744581 6824 raft_consensus.cc:1261] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Refusing update from remote peer b6b7aec192fc4d959a5113bfd2f8d5fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:05.745345 7186 consensus_queue.cc:1035] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:09:05.748675 6906 tablet.cc:2369] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:05.756078 7193 mvcc.cc:205] Tried to move back new op lower bound from 7317299796952621056 to 7317299796743553024. Current Snapshot: MvccSnapshot[applied={T|T < 7317299796952621056}]
I20260811 12:09:05.779927 6318 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:05.780340 6318 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:05.784937 6318 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Sending DeleteTablet for 3 replicas of tablet 3f431aa44e20475dba596e2e05653fef
I20260811 12:09:05.786228 6546 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f431aa44e20475dba596e2e05653fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:05 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 12:09:05.786587 7025 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f431aa44e20475dba596e2e05653fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:05 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:60984
I20260811 12:09:05.786751 6785 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f431aa44e20475dba596e2e05653fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:05 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:51360
I20260811 12:09:05.794370 6318 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:05.796085 6318 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:05.796454 7208 tablet_replica.cc:330] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: stopping tablet replica
I20260811 12:09:05.797206 7208 raft_consensus.cc:2229] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:05.797911 7208 raft_consensus.cc:2258] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:05.798751 7205 tablet_replica.cc:330] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: stopping tablet replica
I20260811 12:09:05.799314 7205 raft_consensus.cc:2229] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:05.800271 7205 raft_consensus.cc:2258] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:05.801543 7208 ts_tablet_manager.cc:1905] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:05.802345 7205 ts_tablet_manager.cc:1905] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:05.807224 7207 tablet_replica.cc:330] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: stopping tablet replica
I20260811 12:09:05.807778 7207 raft_consensus.cc:2229] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:05.808416 7207 raft_consensus.cc:2258] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:05.810365 6546 tablet_service.cc:1467] Processing CreateTablet for tablet d03b66c78a2f4884bb10721ad798dc52 (DEFAULT_TABLE table=test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.810835 6546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d03b66c78a2f4884bb10721ad798dc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.810830 6785 tablet_service.cc:1467] Processing CreateTablet for tablet d03b66c78a2f4884bb10721ad798dc52 (DEFAULT_TABLE table=test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.810824 7025 tablet_service.cc:1467] Processing CreateTablet for tablet d03b66c78a2f4884bb10721ad798dc52 (DEFAULT_TABLE table=test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:05.811250 6785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d03b66c78a2f4884bb10721ad798dc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.811244 7025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d03b66c78a2f4884bb10721ad798dc52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:05.818410 7175 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Bootstrap starting.
I20260811 12:09:05.819640 7174 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Bootstrap starting.
I20260811 12:09:05.820214 7175 tablet_bootstrap.cc:654] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.820741 7205 ts_tablet_manager.cc:1918] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:05.820982 7205 log.cc:1196] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/3f431aa44e20475dba596e2e05653fef
I20260811 12:09:05.821655 7205 ts_tablet_manager.cc:1939] T 3f431aa44e20475dba596e2e05653fef P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting consensus metadata
I20260811 12:09:05.821731 7174 tablet_bootstrap.cc:654] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.821821 7207 ts_tablet_manager.cc:1905] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:05.823546 6294 catalog_manager.cc:4841] TS b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193:36859): tablet 3f431aa44e20475dba596e2e05653fef (table test-table [id=af4e46d4430b4650aafc209efbe73191]) successfully deleted
I20260811 12:09:05.826323 7172 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Bootstrap starting.
I20260811 12:09:05.828598 7172 tablet_bootstrap.cc:654] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:05.830143 7207 ts_tablet_manager.cc:1918] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:05.830456 7207 log.cc:1196] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/3f431aa44e20475dba596e2e05653fef
I20260811 12:09:05.831127 7207 ts_tablet_manager.cc:1939] T 3f431aa44e20475dba596e2e05653fef P aefd28047a54456692699f4a4519808f: Deleting consensus metadata
I20260811 12:09:05.832661 7172 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: No bootstrap required, opened a new log
I20260811 12:09:05.832854 7172 ts_tablet_manager.cc:1397] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260811 12:09:05.833148 6297 catalog_manager.cc:4841] TS aefd28047a54456692699f4a4519808f (127.4.48.195:42343): tablet 3f431aa44e20475dba596e2e05653fef (table test-table [id=af4e46d4430b4650aafc209efbe73191]) successfully deleted
I20260811 12:09:05.837857 7172 raft_consensus.cc:348] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.838230 7172 raft_consensus.cc:374] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.838431 7172 raft_consensus.cc:729] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2314794b2aea416c966d32795c02899e, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.838866 7172 consensus_queue.cc:260] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.839787 7172 ts_tablet_manager.cc:1428] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260811 12:09:05.844744 7208 ts_tablet_manager.cc:1918] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:05.844926 7208 log.cc:1196] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/3f431aa44e20475dba596e2e05653fef
I20260811 12:09:05.845443 7208 ts_tablet_manager.cc:1939] T 3f431aa44e20475dba596e2e05653fef P 2314794b2aea416c966d32795c02899e: Deleting consensus metadata
I20260811 12:09:05.847213 6289 catalog_manager.cc:4841] TS 2314794b2aea416c966d32795c02899e (127.4.48.194:36803): tablet 3f431aa44e20475dba596e2e05653fef (table test-table [id=af4e46d4430b4650aafc209efbe73191]) successfully deleted
I20260811 12:09:05.849128 7175 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: No bootstrap required, opened a new log
I20260811 12:09:05.849298 7175 ts_tablet_manager.cc:1397] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Time spent bootstrapping tablet: real 0.031s user 0.001s sys 0.008s
I20260811 12:09:05.850253 7175 raft_consensus.cc:348] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.850497 7175 raft_consensus.cc:374] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.850633 7175 raft_consensus.cc:729] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefd28047a54456692699f4a4519808f, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.850983 7175 consensus_queue.cc:260] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.851650 7175 ts_tablet_manager.cc:1428] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:09:05.853879 7174 tablet_bootstrap.cc:492] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: No bootstrap required, opened a new log
I20260811 12:09:05.854099 7174 ts_tablet_manager.cc:1397] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent bootstrapping tablet: real 0.035s user 0.003s sys 0.022s
I20260811 12:09:05.854988 7174 raft_consensus.cc:348] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.855192 7174 raft_consensus.cc:374] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:05.855336 7174 raft_consensus.cc:729] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b7aec192fc4d959a5113bfd2f8d5fd, State: Initialized, Role: FOLLOWER
I20260811 12:09:05.855793 7174 consensus_queue.cc:260] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:05.856402 7174 ts_tablet_manager.cc:1428] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260811 12:09:06.011330 6658 tablet.cc:2369] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:06.082439 7180 raft_consensus.cc:482] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:06.082659 7180 raft_consensus.cc:504] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.084345 7180 leader_election.cc:288] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193:36859), aefd28047a54456692699f4a4519808f (127.4.48.195:42343)
I20260811 12:09:06.092954 7170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.085000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:35163 (local address 127.4.48.195:42343)
0811 12:09:06.085171 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.085176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.085691 (+ 515us) server_negotiation.cc:407] Connection header received
0811 12:09:06.085934 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.085941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.086020 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.086085 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:06.086941 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.087841 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.090594 (+ 2753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.091889 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.092027 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.092102 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.092589 (+ 487us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.092706 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:09:06.093063 7215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.084746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36859 (local address 127.4.48.194:52789)
0811 12:09:06.086309 (+ 1563us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.086328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.086523 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.086896 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.086901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.086929 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:06.087435 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.087446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.088628 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.088632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.090235 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.090247 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.092103 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.092109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.092333 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.092374 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.092479 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.092611 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1414,"mutex_wait_us":453,"spinlock_wait_cycles":4352,"thread_start_us":708,"threads_started":1}
I20260811 12:09:06.093021 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.084800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:52789 (local address 127.4.48.193:36859)
0811 12:09:06.084976 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.084981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.086437 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 12:09:06.086681 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.086686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.086765 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.086871 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:06.087640 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.088500 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.090404 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.091837 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.092035 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.092102 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.092524 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.092651 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:09:06.093927 7214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.084742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42343 (local address 127.4.48.194:35163)
0811 12:09:06.085553 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.085575 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.085731 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.086302 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.086305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.086328 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:06.086759 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.086771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.087975 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.087981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.090432 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.090448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.092271 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.092275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.092416 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.092440 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.093254 (+ 814us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.093399 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":2,"spinlock_wait_cycles":17792,"thread_start_us":431,"threads_started":1}
I20260811 12:09:06.094220 6574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d03b66c78a2f4884bb10721ad798dc52" candidate_uuid: "2314794b2aea416c966d32795c02899e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" is_pre_election: true
I20260811 12:09:06.094728 6574 raft_consensus.cc:2454] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2314794b2aea416c966d32795c02899e in term 0.
I20260811 12:09:06.094714 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d03b66c78a2f4884bb10721ad798dc52" candidate_uuid: "2314794b2aea416c966d32795c02899e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f" is_pre_election: true
I20260811 12:09:06.094990 7058 raft_consensus.cc:2454] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2314794b2aea416c966d32795c02899e in term 0.
I20260811 12:09:06.095377 6689 leader_election.cc:302] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2314794b2aea416c966d32795c02899e, b6b7aec192fc4d959a5113bfd2f8d5fd; no voters:
I20260811 12:09:06.095772 7180 raft_consensus.cc:2781] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:06.095906 7180 raft_consensus.cc:482] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:06.096019 7180 raft_consensus.cc:3037] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.098290 7180 raft_consensus.cc:504] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.098907 7180 leader_election.cc:288] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [CANDIDATE]: Term 1 election: Requested vote from peers b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193:36859), aefd28047a54456692699f4a4519808f (127.4.48.195:42343)
I20260811 12:09:06.099378 6574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d03b66c78a2f4884bb10721ad798dc52" candidate_uuid: "2314794b2aea416c966d32795c02899e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd"
I20260811 12:09:06.099495 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d03b66c78a2f4884bb10721ad798dc52" candidate_uuid: "2314794b2aea416c966d32795c02899e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f"
I20260811 12:09:06.099607 6574 raft_consensus.cc:3037] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.099715 7058 raft_consensus.cc:3037] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.102746 7058 raft_consensus.cc:2454] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2314794b2aea416c966d32795c02899e in term 1.
I20260811 12:09:06.103178 6574 raft_consensus.cc:2454] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2314794b2aea416c966d32795c02899e in term 1.
I20260811 12:09:06.103181 6692 leader_election.cc:302] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2314794b2aea416c966d32795c02899e, aefd28047a54456692699f4a4519808f; no voters:
I20260811 12:09:06.103518 7180 raft_consensus.cc:2781] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:06.104044 7180 raft_consensus.cc:686] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 1 LEADER]: Becoming Leader. State: Replica: 2314794b2aea416c966d32795c02899e, State: Running, Role: LEADER
I20260811 12:09:06.104542 7180 consensus_queue.cc:237] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.108465 6317 catalog_manager.cc:5495] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e reported cstate change: term changed from 0 to 1, leader changed from <none> to 2314794b2aea416c966d32795c02899e (127.4.48.194). New cstate: current_term: 1 leader_uuid: "2314794b2aea416c966d32795c02899e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450143, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:06.174039 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.158393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58688 (local address 127.4.48.254:38047)
0811 12:09:06.159589 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.159593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.159609 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:06.159651 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.159654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.159758 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.159877 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.161125 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.162072 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.163470 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.164058 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.164173 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.168114 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.168119 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.168122 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.168151 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.170821 (+ 2670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.171789 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.171793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.171795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.171918 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.172415 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.172419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.172421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.172687 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.172792 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.173380 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.173719 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":149,"server-negotiator.queue_time_us":1080,"thread_start_us":901,"threads_started":1}
I20260811 12:09:06.174108 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.158348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38047 (local address 127.0.0.1:58688)
0811 12:09:06.159205 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.159228 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.159401 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.159910 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.159915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.160710 (+ 795us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:06.160966 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.160977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.162186 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.162191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.163334 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.163347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.164227 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.164233 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.164387 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.165034 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:06.165051 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:06.167920 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:06.170986 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:06.171000 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:06.171010 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:06.171650 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:06.172026 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:06.172031 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:06.172035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:06.172288 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:06.172826 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:06.172835 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:06.173052 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.173279 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.173594 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":683,"thread_start_us":503,"threads_started":1}
I20260811 12:09:06.180253 6317 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58688:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:09:06.181595 6317 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:06.188463 6546 tablet_service.cc:1467] Processing CreateTablet for tablet 555fba8c49854aa28ed341621fab3eeb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be239ee3d0104471b04b8c36d4938b27]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:06.188931 6785 tablet_service.cc:1467] Processing CreateTablet for tablet 555fba8c49854aa28ed341621fab3eeb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be239ee3d0104471b04b8c36d4938b27]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:06.189199 6546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 555fba8c49854aa28ed341621fab3eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:06.189091 7025 tablet_service.cc:1467] Processing CreateTablet for tablet 555fba8c49854aa28ed341621fab3eeb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be239ee3d0104471b04b8c36d4938b27]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:06.189492 6785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 555fba8c49854aa28ed341621fab3eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:06.189685 7025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 555fba8c49854aa28ed341621fab3eeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:06.197747 7175 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: Bootstrap starting.
I20260811 12:09:06.199752 7175 tablet_bootstrap.cc:654] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:06.200294 7174 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: Bootstrap starting.
I20260811 12:09:06.201993 7174 tablet_bootstrap.cc:654] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:06.204345 7172 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: Bootstrap starting.
I20260811 12:09:06.204918 7174 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: No bootstrap required, opened a new log
I20260811 12:09:06.205106 7174 ts_tablet_manager.cc:1397] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 12:09:06.206205 7172 tablet_bootstrap.cc:654] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:06.205929 7174 raft_consensus.cc:348] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.206593 7174 raft_consensus.cc:374] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:06.206699 7174 raft_consensus.cc:729] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6b7aec192fc4d959a5113bfd2f8d5fd, State: Initialized, Role: FOLLOWER
I20260811 12:09:06.207031 7174 consensus_queue.cc:260] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.207608 7186 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.207942 7186 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.213673 7174 ts_tablet_manager.cc:1428] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260811 12:09:06.216992 7175 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: No bootstrap required, opened a new log
I20260811 12:09:06.217258 7175 ts_tablet_manager.cc:1397] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: Time spent bootstrapping tablet: real 0.020s user 0.012s sys 0.004s
I20260811 12:09:06.217653 7172 tablet_bootstrap.cc:492] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: No bootstrap required, opened a new log
I20260811 12:09:06.217962 7172 ts_tablet_manager.cc:1397] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260811 12:09:06.218155 7175 raft_consensus.cc:348] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.218432 7175 raft_consensus.cc:374] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:06.218528 7175 raft_consensus.cc:729] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aefd28047a54456692699f4a4519808f, State: Initialized, Role: FOLLOWER
I20260811 12:09:06.218787 7175 consensus_queue.cc:260] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.218910 7172 raft_consensus.cc:348] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.219177 7172 raft_consensus.cc:374] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:06.219271 7172 raft_consensus.cc:729] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2314794b2aea416c966d32795c02899e, State: Initialized, Role: FOLLOWER
I20260811 12:09:06.219267 7181 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.219637 7181 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.219542 7172 consensus_queue.cc:260] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.220186 7172 ts_tablet_manager.cc:1428] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:09:06.220074 7180 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.220407 7180 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.220978 7175 ts_tablet_manager.cc:1428] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 12:09:06.307427 7186 raft_consensus.cc:482] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:06.307608 7186 raft_consensus.cc:504] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.308254 7186 leader_election.cc:288] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2314794b2aea416c966d32795c02899e (127.4.48.194:36803), aefd28047a54456692699f4a4519808f (127.4.48.195:42343)
I20260811 12:09:06.308768 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "555fba8c49854aa28ed341621fab3eeb" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f" is_pre_election: true
I20260811 12:09:06.308808 6824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "555fba8c49854aa28ed341621fab3eeb" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2314794b2aea416c966d32795c02899e" is_pre_election: true
I20260811 12:09:06.309020 7058 raft_consensus.cc:2454] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 0.
I20260811 12:09:06.309029 6824 raft_consensus.cc:2454] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 0.
I20260811 12:09:06.309437 6468 leader_election.cc:302] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aefd28047a54456692699f4a4519808f, b6b7aec192fc4d959a5113bfd2f8d5fd; no voters:
I20260811 12:09:06.309746 7186 raft_consensus.cc:2781] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:06.309880 7186 raft_consensus.cc:482] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:06.309969 7186 raft_consensus.cc:3037] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.312109 7186 raft_consensus.cc:504] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.312585 7186 leader_election.cc:288] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 election: Requested vote from peers 2314794b2aea416c966d32795c02899e (127.4.48.194:36803), aefd28047a54456692699f4a4519808f (127.4.48.195:42343)
I20260811 12:09:06.312988 7058 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "555fba8c49854aa28ed341621fab3eeb" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aefd28047a54456692699f4a4519808f"
I20260811 12:09:06.313017 6824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "555fba8c49854aa28ed341621fab3eeb" candidate_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2314794b2aea416c966d32795c02899e"
I20260811 12:09:06.313161 7058 raft_consensus.cc:3037] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.313192 6824 raft_consensus.cc:3037] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:06.315037 7058 raft_consensus.cc:2454] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 1.
I20260811 12:09:06.315213 6824 raft_consensus.cc:2454] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6b7aec192fc4d959a5113bfd2f8d5fd in term 1.
I20260811 12:09:06.315399 6468 leader_election.cc:302] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aefd28047a54456692699f4a4519808f, b6b7aec192fc4d959a5113bfd2f8d5fd; no voters:
I20260811 12:09:06.315647 7186 raft_consensus.cc:2781] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:06.316026 7186 raft_consensus.cc:686] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 LEADER]: Becoming Leader. State: Replica: b6b7aec192fc4d959a5113bfd2f8d5fd, State: Running, Role: LEADER
I20260811 12:09:06.316288 7186 consensus_queue.cc:237] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } }
I20260811 12:09:06.318367 7239 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: TxnStatusTablet state changed. Reason: New leader b6b7aec192fc4d959a5113bfd2f8d5fd. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.318718 7239 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:06.319056 6318 catalog_manager.cc:5495] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd reported cstate change: term changed from 0 to 1, leader changed from <none> to b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193). New cstate: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } health_report { overall_health: UNKNOWN } } }
I20260811 12:09:06.319408 7241 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:09:06.319775 7241 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:09:06.364672 7228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.358942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36859 (local address 127.0.0.1:45934)
0811 12:09:06.359185 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.359206 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.359329 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.359764 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.359768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.359778 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:06.359955 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.359963 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.361388 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.361392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.362469 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.362481 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.363459 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.363463 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.363625 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.363714 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.364115 (+ 401us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.364136 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.364206 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.364360 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:09:06.364672 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.358983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45934 (local address 127.4.48.193:36859)
0811 12:09:06.359183 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.359189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.359270 (+ 81us) server_negotiation.cc:407] Connection header received
0811 12:09:06.359507 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.359513 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.359606 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.359742 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:06.360096 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.361250 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.362596 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.363304 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.363440 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.363886 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.364021 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.364100 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.364247 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.364330 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 12:09:06.368937 6824 raft_consensus.cc:1261] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Refusing update from remote peer b6b7aec192fc4d959a5113bfd2f8d5fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:06.369053 7058 raft_consensus.cc:1261] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Refusing update from remote peer b6b7aec192fc4d959a5113bfd2f8d5fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:06.369585 7239 consensus_queue.cc:1035] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:06.369947 7186 consensus_queue.cc:1035] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:06.374266 7181 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: TxnStatusTablet state changed. Reason: New leader b6b7aec192fc4d959a5113bfd2f8d5fd. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.374549 7181 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.374962 7180 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: TxnStatusTablet state changed. Reason: New leader b6b7aec192fc4d959a5113bfd2f8d5fd. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.375242 7180 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.376607 7246 mvcc.cc:205] Tried to move back new op lower bound from 7317299799521423360 to 7317299799313584128. Current Snapshot: MvccSnapshot[applied={T|T < 7317299799521423360}]
I20260811 12:09:06.382678 7181 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.382947 7181 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P aefd28047a54456692699f4a4519808f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.384645 7239 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.384941 7239 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:06.387152 7239 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.387424 7239 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P b6b7aec192fc4d959a5113bfd2f8d5fd: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:06.388232 7180 tablet_replica.cc:439] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 } } peers { permanent_uuid: "2314794b2aea416c966d32795c02899e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36803 } } peers { permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 } } }
I20260811 12:09:06.388526 7180 tablet_replica.cc:442] T 555fba8c49854aa28ed341621fab3eeb P 2314794b2aea416c966d32795c02899e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:06.407444 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.393328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51366 (local address 127.4.48.194:36803)
0811 12:09:06.393874 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.393878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.401961 (+ 8083us) server_negotiation.cc:407] Connection header received
0811 12:09:06.402150 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.402154 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.402226 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.402387 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:06.402851 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.403730 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.405138 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.405613 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.405719 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.406167 (+ 448us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.406781 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.406912 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.407038 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.407116 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":317,"threads_started":1}
I20260811 12:09:06.425132 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.418240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:39655 (local address 127.4.48.193:36859)
0811 12:09:06.418445 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.418452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.419187 (+ 735us) server_negotiation.cc:407] Connection header received
0811 12:09:06.419401 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.419404 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.419467 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.419536 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:06.420311 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.421091 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.423272 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.423785 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.423884 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.424446 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.424567 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.424630 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.424777 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.424836 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:06.425381 7264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.418192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36859 (local address 127.4.48.194:39655)
0811 12:09:06.419077 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.419102 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.419279 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.419576 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.419582 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.419596 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:06.420131 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.420145 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.421235 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.421241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.423114 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.423131 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.423974 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.423982 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.424200 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.424284 (+ 84us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.424661 (+ 377us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.424673 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.424758 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.424899 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"thread_start_us":446,"threads_started":1}
I20260811 12:09:06.436381 7058 raft_consensus.cc:1261] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Refusing update from remote peer 2314794b2aea416c966d32795c02899e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:06.437139 7216 consensus_queue.cc:1035] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [LEADER]: Connected to new peer: Peer: permanent_uuid: "aefd28047a54456692699f4a4519808f" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:06.444169 7247 mvcc.cc:205] Tried to move back new op lower bound from 7317299799796805632 to 7317299798448812032. Current Snapshot: MvccSnapshot[applied={T|T < 7317299799796805632}]
I20260811 12:09:06.448698 6574 raft_consensus.cc:1261] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Refusing update from remote peer 2314794b2aea416c966d32795c02899e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:06.449921 7180 consensus_queue.cc:1035] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6b7aec192fc4d959a5113bfd2f8d5fd" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36859 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:06.474628 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.467139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37387 (local address 127.4.48.194:36803)
0811 12:09:06.467321 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.467328 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.468144 (+ 816us) server_negotiation.cc:407] Connection header received
0811 12:09:06.468340 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.468346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.468413 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.468484 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:06.469195 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.470445 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.472092 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.472971 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.473100 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.473746 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.473858 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.473932 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.474266 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.474344 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:09:06.474846 7270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.467053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36803 (local address 127.4.48.193:37387)
0811 12:09:06.468018 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:06.468049 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:06.468211 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:06.468517 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:06.468522 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:06.468535 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:06.468992 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.469006 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.470561 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.470565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.471941 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:06.471954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.473306 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.473315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:06.473492 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.473577 (+ 85us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:06.473955 (+ 378us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:06.473967 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:09:06.474077 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 12:09:06.474238 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":545,"threads_started":1}
I20260811 12:09:06.530371 6317 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:06.530663 6317 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:06.534752 6317 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6fbe4f44876d471d8cae505e2524354b: Sending DeleteTablet for 3 replicas of tablet d03b66c78a2f4884bb10721ad798dc52
I20260811 12:09:06.536376 6546 tablet_service.cc:1514] Processing DeleteTablet for tablet d03b66c78a2f4884bb10721ad798dc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:06 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:45928
I20260811 12:09:06.536897 7025 tablet_service.cc:1514] Processing DeleteTablet for tablet d03b66c78a2f4884bb10721ad798dc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:06 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:60984
I20260811 12:09:06.537895 6785 tablet_service.cc:1514] Processing DeleteTablet for tablet d03b66c78a2f4884bb10721ad798dc52 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:06 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:51360
I20260811 12:09:06.544137 7275 tablet_replica.cc:330] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: stopping tablet replica
I20260811 12:09:06.544358 7275 raft_consensus.cc:2229] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:06.544628 7275 raft_consensus.cc:2258] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:06.545827 7275 ts_tablet_manager.cc:1905] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:06.553048 7275 ts_tablet_manager.cc:1918] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:06.553179 7275 log.cc:1196] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/d03b66c78a2f4884bb10721ad798dc52
I20260811 12:09:06.553625 7275 ts_tablet_manager.cc:1939] T d03b66c78a2f4884bb10721ad798dc52 P b6b7aec192fc4d959a5113bfd2f8d5fd: Deleting consensus metadata
I20260811 12:09:06.554677 6294 catalog_manager.cc:4841] TS b6b7aec192fc4d959a5113bfd2f8d5fd (127.4.48.193:36859): tablet d03b66c78a2f4884bb10721ad798dc52 (table test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]) successfully deleted
I20260811 12:09:06.556591 7278 tablet_replica.cc:330] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: stopping tablet replica
I20260811 12:09:06.556813 7278 raft_consensus.cc:2229] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:06.557439 7278 raft_consensus.cc:2258] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:06.558033 7277 tablet_replica.cc:330] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: stopping tablet replica
I20260811 12:09:06.558234 7277 raft_consensus.cc:2229] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:06.558513 7277 raft_consensus.cc:2258] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:06.559738 7277 ts_tablet_manager.cc:1905] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:06.566788 7277 ts_tablet_manager.cc:1918] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:06.566921 7277 log.cc:1196] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/d03b66c78a2f4884bb10721ad798dc52
I20260811 12:09:06.567385 7277 ts_tablet_manager.cc:1939] T d03b66c78a2f4884bb10721ad798dc52 P aefd28047a54456692699f4a4519808f: Deleting consensus metadata
I20260811 12:09:06.568467 6297 catalog_manager.cc:4841] TS aefd28047a54456692699f4a4519808f (127.4.48.195:42343): tablet d03b66c78a2f4884bb10721ad798dc52 (table test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]) successfully deleted
I20260811 12:09:06.569392 7278 ts_tablet_manager.cc:1905] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:06.576722 7278 ts_tablet_manager.cc:1918] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:06.576871 7278 log.cc:1196] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/d03b66c78a2f4884bb10721ad798dc52
I20260811 12:09:06.577358 7278 ts_tablet_manager.cc:1939] T d03b66c78a2f4884bb10721ad798dc52 P 2314794b2aea416c966d32795c02899e: Deleting consensus metadata
I20260811 12:09:06.578465 6289 catalog_manager.cc:4841] TS 2314794b2aea416c966d32795c02899e (127.4.48.194:36803): tablet d03b66c78a2f4884bb10721ad798dc52 (table test-table [id=1e1a7157e5014045b1bbfc0ceb9373d8]) successfully deleted
I20260811 12:09:06.583376 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.557383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45940 (local address 127.4.48.193:36859)
0811 12:09:06.557572 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.557577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.565094 (+ 7517us) server_negotiation.cc:407] Connection header received
0811 12:09:06.565573 (+ 479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.565579 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.565657 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.565728 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.567945 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.569032 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.570799 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.571407 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.571509 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.576662 (+ 5153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.576679 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.576700 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.576760 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.580425 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.581213 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.581222 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.581224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.581379 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.581785 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.581790 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.581797 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.582185 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.582483 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.582774 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.583001 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 12:09:06.584360 6647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45940
I20260811 12:09:06.600666 7230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.587651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.4.48.254:38047)
0811 12:09:06.587849 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.587853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.588440 (+ 587us) server_negotiation.cc:407] Connection header received
0811 12:09:06.588588 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.588594 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.588666 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.588745 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.590087 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.590870 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.592128 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.592693 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.592811 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.595380 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.595385 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.595388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.595421 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.598103 (+ 2682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.598701 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.598705 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.598707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.598843 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.599202 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.599206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.599208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.599509 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.599785 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.600123 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.600362 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 12:09:06.601524 6317 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58700
I20260811 12:09:06.603525 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6443
I20260811 12:09:06.618471 7166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.606175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.4.48.193:36859)
0811 12:09:06.606350 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.606356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.606373 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:09:06.606524 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.606530 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.606603 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.606664 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.607916 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.608668 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.609855 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.610584 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.610706 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.613063 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.613069 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.613072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.613104 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.615878 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.616446 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.616450 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.616453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.616606 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.616947 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.616951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.616953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.617217 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.617482 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.617745 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.617982 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 12:09:06.619031 6647 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45944
W20260811 12:09:06.619647 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6443: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:06.619861 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6443
I20260811 12:09:06.657500 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6661
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:06.673281 7258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.658231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.4.48.194:36803)
0811 12:09:06.658420 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.658425 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.658445 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:09:06.658661 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.658666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.658746 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.658826 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.660386 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.661270 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.662823 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.663298 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.663422 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.667596 (+ 4174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.667609 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.667626 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.667660 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.670626 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.671252 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.671260 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.671263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.671403 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.671779 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.671784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.671787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.672156 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.672490 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.672761 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.672994 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:09:06.674024 6891 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51382
W20260811 12:09:06.674676 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:06.674849 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6661
I20260811 12:09:06.697582 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6910
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[6238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450145, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:06.711230 7287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:06.698295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60998 (local address 127.4.48.195:42343)
0811 12:09:06.698797 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:06.698801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:06.698818 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:09:06.698870 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:06.698874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:06.698931 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:06.699017 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:06.700196 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.700931 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.702041 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:06.702438 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:06.702520 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:06.706332 (+ 3812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:06.706350 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:06.706369 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:06.706402 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:06.709066 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.709654 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.709661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.709663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.709791 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:06.710069 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:06.710072 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:06.710074 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:06.710395 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:06.710608 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:06.710802 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 12:09:06.710977 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":242,"threads_started":1}
W20260811 12:09:06.711939 7128 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60998
W20260811 12:09:06.712527 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6910: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:06.712656 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6910
I20260811 12:09:06.735656 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6263
I20260811 12:09:06.736881 6393 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:06.850876 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6263
2026-08-11T12:09:06Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4827 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[7293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[7293](info): set up 2 sockets
Aug 11 12:09:06 dist-test-slave-nj21 krb5kdc[7293](info): commencing operation
krb5kdc: starting...
W20260811 12:09:08.068296 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.165s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:08 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:08.252552 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36117
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:08.568589 7315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:08.568859 7315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:08.569003 7315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:08.575960 7315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:08.576061 7315 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:09:08.576131 7315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:08.576218 7315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:08.576295 7315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:08.576380 7315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:08.585750 7315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:36117
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:08.586936 7315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:08.588225 7315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:08.599736 7330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:08.599807 7333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:08.599772 7339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:08.600906 7315 server_base.cc:1034] running on GCE node
I20260811 12:09:08.601588 7315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:08.602864 7315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:08.604074 7315 hybrid_clock.cc:648] HybridClock initialized: now 1786450148604003 us; error 38 us; skew 500 ppm
I20260811 12:09:08.604393 7315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:08 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:08.607550 7315 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:08.608939 7315 webserver.cc:460] Webserver started at http://127.4.48.254:44669/ using document root <none> and password file <none>
I20260811 12:09:08.609580 7315 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:08.609666 7315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:08.609941 7315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:08.612870 7315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "0162b84de83c4e4380d6632a1c8e224c"
format_stamp: "Formatted at 2026-08-11 12:09:08 on dist-test-slave-nj21"
server_key: "fe4855eb311f49d08216b016d9834761"
server_key_iv: "461b54272c2081b450099731a10c8d50"
server_key_version: "encryptionkey@0"
I20260811 12:09:08.613579 7315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "0162b84de83c4e4380d6632a1c8e224c"
format_stamp: "Formatted at 2026-08-11 12:09:08 on dist-test-slave-nj21"
server_key: "fe4855eb311f49d08216b016d9834761"
server_key_iv: "461b54272c2081b450099731a10c8d50"
server_key_version: "encryptionkey@0"
I20260811 12:09:08.618398 7315 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:09:08.621628 7351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:08.622865 7315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:08.623026 7315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "0162b84de83c4e4380d6632a1c8e224c"
format_stamp: "Formatted at 2026-08-11 12:09:08 on dist-test-slave-nj21"
server_key: "fe4855eb311f49d08216b016d9834761"
server_key_iv: "461b54272c2081b450099731a10c8d50"
server_key_version: "encryptionkey@0"
I20260811 12:09:08.623215 7315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:08.635089 7315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:08.639168 7315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:08.639412 7315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:08.660146 7315 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:36117
I20260811 12:09:08.660292 7452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:36117 every 8 connection(s)
I20260811 12:09:08.661577 7315 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:08.665385 7455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:08.667331 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7315
I20260811 12:09:08.667526 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:08.668201 4291 external_mini_cluster.cc:1428] Setting key d4627fc11b3563faa83c9a3cf3a96d4b
I20260811 12:09:08.677034 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Bootstrap starting.
I20260811 12:09:08.680805 7455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:08.682125 7455 log.cc:824] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:09:08 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:08.685019 7455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: No bootstrap required, opened a new log
I20260811 12:09:08.690760 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:08.670122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.4.48.254:36117)
0811 12:09:08.670920 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:08.670933 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:08.670974 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:09:08.672680 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:08.672704 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:08.673061 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:08.673427 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:08.675170 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:08.676503 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:08.678207 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:08.678886 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:08.679009 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:08.683832 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:08.683841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:08.683855 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:08.683891 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:08.686796 (+ 2905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:08.687454 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:08.687461 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:08.687468 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:08.687646 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:08.688043 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:08.688047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:08.688049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:08.688579 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:08.688865 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:08.689334 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 12:09:08.689747 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":368,"threads_started":1}
I20260811 12:09:08.692740 7455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } }
I20260811 12:09:08.693099 7455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:08.693207 7455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0162b84de83c4e4380d6632a1c8e224c, State: Initialized, Role: FOLLOWER
I20260811 12:09:08.693811 7455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } }
I20260811 12:09:08.694036 7455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:08.694133 7455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:08.694257 7455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:08.696334 7455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } }
I20260811 12:09:08.696791 7455 leader_election.cc:302] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0162b84de83c4e4380d6632a1c8e224c; no voters:
I20260811 12:09:08.697290 7455 leader_election.cc:288] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:08.697418 7464 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:08.698506 7464 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [term 1 LEADER]: Becoming Leader. State: Replica: 0162b84de83c4e4380d6632a1c8e224c, State: Running, Role: LEADER
I20260811 12:09:08.699079 7464 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } }
I20260811 12:09:08.699388 7455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:08.701849 7466 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0162b84de83c4e4380d6632a1c8e224c. Latest consensus state: current_term: 1 leader_uuid: "0162b84de83c4e4380d6632a1c8e224c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } } }
I20260811 12:09:08.701880 7465 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0162b84de83c4e4380d6632a1c8e224c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0162b84de83c4e4380d6632a1c8e224c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36117 } } }
I20260811 12:09:08.702255 7466 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:08.702257 7465 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:08.707497 7478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:08.712750 7478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:08.721763 7478 catalog_manager.cc:1347] Generated new cluster ID: fe9c6f6580af4d7593e3d9d6082d0396
I20260811 12:09:08.721885 7478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:08.751609 7478 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:08.752946 7478 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:08.758095 7478 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Generated new TSK 0
I20260811 12:09:08.759032 7478 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:08.814775 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36117
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:09.138190 7500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:09.138473 7500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:09.138621 7500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:09.145761 7500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:09.145866 7500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:09.145990 7500 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:09:09.146054 7500 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:09.146153 7500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:09.155709 7500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:09.156956 7500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:09.158272 7500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:09.171056 7509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:09.171087 7512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:09.171133 7515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:09.171828 7500 server_base.cc:1034] running on GCE node
I20260811 12:09:09.172429 7500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:09.173321 7500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:09.174525 7500 hybrid_clock.cc:648] HybridClock initialized: now 1786450149174475 us; error 44 us; skew 500 ppm
I20260811 12:09:09.174824 7500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:09 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:09.188136 7500 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:09.190114 7500 webserver.cc:460] Webserver started at http://127.4.48.193:46843/ using document root <none> and password file <none>
I20260811 12:09:09.191022 7500 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:09.191162 7500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:09.191530 7500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:09.197335 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "86e003343208903abf6219245a18f66c"
server_key_iv: "73b2255a7362396fefe1944a7a9a143e"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.198056 7500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "86e003343208903abf6219245a18f66c"
server_key_iv: "73b2255a7362396fefe1944a7a9a143e"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.203286 7500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:09:09.206571 7532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:09.208010 7500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:09:09.208137 7500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "86e003343208903abf6219245a18f66c"
server_key_iv: "73b2255a7362396fefe1944a7a9a143e"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.208288 7500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:09.230594 7500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:09.234421 7500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:09.234671 7500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:09.237406 7500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:09.237500 7500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:09.237615 7500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:09.237699 7500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:09 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:09.259415 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.237843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:60997 (local address 127.4.48.254:36117)
0811 12:09:09.238601 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:09.238608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:09.239607 (+ 999us) server_negotiation.cc:407] Connection header received
0811 12:09:09.241634 (+ 2027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:09.241640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:09.241724 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:09.241819 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:09.243993 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.244958 (+ 965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.247369 (+ 2411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.247959 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.248074 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.252076 (+ 4002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:09.252083 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:09.252087 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:09.252127 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:09.255580 (+ 3453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.256589 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.256595 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.256599 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.256775 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.257341 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.257347 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.257350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.257760 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:09.257998 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:09.258735 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 12:09:09.258983 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":528,"threads_started":1}
I20260811 12:09:09.260092 7547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.238490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.193:60997)
0811 12:09:09.239432 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:09.239489 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:09.241375 (+ 1886us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:09.242115 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:09.242126 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:09.242906 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:09.243734 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.243772 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.245115 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.245121 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.247221 (+ 2100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.247233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.248205 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.248213 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.248493 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.249255 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:09.249295 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:09.251822 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:09.255744 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.255752 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.255769 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.256436 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.256907 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.256912 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.256916 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.257220 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.257968 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:09.257976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:09.258369 (+ 393us) client_negotiation.cc:769] Sending connection context
0811 12:09:09.258595 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 12:09:09.258967 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":402,"threads_started":1}
I20260811 12:09:09.277534 7500 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:34747
I20260811 12:09:09.277555 7736 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:34747 every 8 connection(s)
I20260811 12:09:09.278774 7500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:09.280329 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7500
I20260811 12:09:09.280550 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:09.280876 7685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.267220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.193:56539)
0811 12:09:09.267914 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:09.267937 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:09.268084 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:09.268471 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:09.268476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:09.269144 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:09.269550 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.269562 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.270899 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.270904 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.272454 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.272466 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.273506 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.273510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.273685 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.274326 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:09.274349 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:09.275411 (+ 1062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:09.278377 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.278382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.278386 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.278767 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.279110 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.279115 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.279119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.279244 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.279842 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:09.279859 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:09.280059 (+ 200us) client_negotiation.cc:769] Sending connection context
0811 12:09:09.280192 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 12:09:09.280470 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":377,"threads_started":1}
I20260811 12:09:09.281375 4291 external_mini_cluster.cc:1428] Setting key acca291e1822ba109548330e7032dc46
I20260811 12:09:09.281412 7737 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36117
I20260811 12:09:09.281623 7737 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:09.282104 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.267747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56539 (local address 127.4.48.254:36117)
0811 12:09:09.267921 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:09.267926 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:09.268017 (+ 91us) server_negotiation.cc:407] Connection header received
0811 12:09:09.268230 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:09.268236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:09.268318 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:09.268424 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:09.269696 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.270739 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.272639 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.273158 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.273280 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.275640 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:09.275647 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:09.275652 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:09.275710 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:09.278252 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.278879 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.278885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.278888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.279009 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.279345 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.279350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.279354 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.279696 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:09.281448 (+ 1752us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:09.281578 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 12:09:09.281791 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:09:09.282413 7737 heartbeater.cc:511] Master 127.4.48.254:36117 requested a full tablet report, sending...
I20260811 12:09:09.284464 7378 ts_manager.cc:194] Registered new tserver with Master: f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747)
I20260811 12:09:09.286563 7378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:60997
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:09.324579 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36117
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:09.639394 7748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:09.639688 7748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:09.639850 7748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:09.646925 7748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:09.647039 7748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:09.647167 7748 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:09:09.647234 7748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:09.647338 7748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:09.656713 7748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:09.657963 7748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:09.659416 7748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:09.673503 7760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:09.673473 7764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:09.679919 7748 server_base.cc:1034] running on GCE node
W20260811 12:09:09.680859 7769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:09.681571 7748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:09.682614 7748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:09.683804 7748 hybrid_clock.cc:648] HybridClock initialized: now 1786450149683758 us; error 43 us; skew 500 ppm
I20260811 12:09:09.684084 7748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:09 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:09.687055 7748 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:09.688498 7748 webserver.cc:460] Webserver started at http://127.4.48.194:34853/ using document root <none> and password file <none>
I20260811 12:09:09.689147 7748 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:09.689261 7748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:09.689568 7748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:09.692375 7748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "96e8f25b2a734d8ea31683ac79882de1"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "945769af99e9cc5c4cb6a8026dbbf5b4"
server_key_iv: "ac12ef86c4de34a0ed17e76c4fc51a40"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.693089 7748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "96e8f25b2a734d8ea31683ac79882de1"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "945769af99e9cc5c4cb6a8026dbbf5b4"
server_key_iv: "ac12ef86c4de34a0ed17e76c4fc51a40"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.697861 7748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:09.701197 7778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:09.702391 7748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:09:09.702528 7748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "96e8f25b2a734d8ea31683ac79882de1"
format_stamp: "Formatted at 2026-08-11 12:09:09 on dist-test-slave-nj21"
server_key: "945769af99e9cc5c4cb6a8026dbbf5b4"
server_key_iv: "ac12ef86c4de34a0ed17e76c4fc51a40"
server_key_version: "encryptionkey@0"
I20260811 12:09:09.702687 7748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:09.720240 7748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:09.724025 7748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:09.724279 7748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:09.726950 7748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:09.727051 7748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:09.727166 7748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:09.727243 7748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:09 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450149, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:09.747624 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.727256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:49619 (local address 127.4.48.254:36117)
0811 12:09:09.727446 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:09.727452 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:09.728938 (+ 1486us) server_negotiation.cc:407] Connection header received
0811 12:09:09.730987 (+ 2049us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:09.730994 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:09.731090 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:09.731197 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:09.733258 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.734383 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.736039 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.736649 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.736782 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.741098 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:09.741104 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:09.741107 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:09.741141 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:09.743933 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.744815 (+ 882us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.744821 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.744826 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.744966 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.745592 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.745598 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.745602 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.745979 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:09.746128 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:09.747171 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0811 12:09:09.747379 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:09:09.749006 7796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.727911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.194:49619)
0811 12:09:09.728740 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:09.728795 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:09.730700 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:09.731442 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:09.731452 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:09.732210 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:09.733009 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.733032 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.734565 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.734571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.735890 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.735906 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.737027 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.737037 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.737328 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.737994 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:09.738019 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:09.740868 (+ 2849us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:09.744093 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.744102 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.744116 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.744688 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.745095 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.745100 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.745103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.745415 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.746207 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:09.746229 (+ 22us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:09.746698 (+ 469us) client_negotiation.cc:769] Sending connection context
0811 12:09:09.747034 (+ 336us) client_negotiation.cc:240] Negotiation successful
0811 12:09:09.747583 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":347,"threads_started":1}
I20260811 12:09:09.774966 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.761932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:40389 (local address 127.4.48.254:36117)
0811 12:09:09.762097 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:09.762102 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:09.762638 (+ 536us) server_negotiation.cc:407] Connection header received
0811 12:09:09.762834 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:09.762839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:09.762923 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:09.763008 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:09.763978 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.764821 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.766516 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.767170 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.767315 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.769341 (+ 2026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:09.769347 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:09.769352 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:09.769390 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:09.772443 (+ 3053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.773059 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.773065 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.773068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.773202 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:09.773547 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:09.773553 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:09.773556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:09.773881 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:09.774010 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:09.774416 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 12:09:09.774629 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:09:09.775048 7934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:09.761975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.194:40389)
0811 12:09:09.762544 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:09.762559 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:09.762682 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:09.763038 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:09.763042 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:09.763500 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:09.763841 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.763859 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.764962 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.764966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.766362 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:09.766382 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:09.767346 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:09.767352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:09.767471 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:09.768055 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:09.768083 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:09.769157 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:09.772560 (+ 3403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.772566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.772570 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.772948 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.773305 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:09.773310 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:09.773314 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:09.773446 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:09.773991 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:09.773996 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:09.774164 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 12:09:09.774295 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:09:09.774604 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":320,"threads_started":1}
I20260811 12:09:09.776542 7748 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:45419
I20260811 12:09:09.776546 7973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:45419 every 8 connection(s)
I20260811 12:09:09.778172 7748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:09.780383 7974 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36117
I20260811 12:09:09.780572 7974 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:09.781229 7974 heartbeater.cc:511] Master 127.4.48.254:36117 requested a full tablet report, sending...
I20260811 12:09:09.781409 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7748
I20260811 12:09:09.781559 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:09.782007 4291 external_mini_cluster.cc:1428] Setting key be7d4385b3c3e676669c82284791df9e
I20260811 12:09:09.782593 7378 ts_manager.cc:194] Registered new tserver with Master: 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:09.783435 7378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:49619
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:09.824568 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36117
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34187
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:10.139662 7981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:10.140062 7981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:10.140218 7981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:10.147274 7981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:10.147375 7981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:10.147500 7981 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 12:09:10.147562 7981 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:10.147663 7981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:10.157352 7981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34187
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:10.158601 7981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:10.159966 7981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:10.172355 7986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:10.172252 7990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:10.172436 7987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:10.173295 7981 server_base.cc:1034] running on GCE node
I20260811 12:09:10.173863 7981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:10.174696 7981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:10.175891 7981 hybrid_clock.cc:648] HybridClock initialized: now 1786450150175835 us; error 46 us; skew 500 ppm
I20260811 12:09:10.176182 7981 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:10 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:10.179073 7981 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:10.180454 7981 webserver.cc:460] Webserver started at http://127.4.48.195:42961/ using document root <none> and password file <none>
I20260811 12:09:10.181090 7981 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:10.181200 7981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:10.181493 7981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:10.184259 7981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "e7be115158c640c984fb06cdaf10fcb9"
format_stamp: "Formatted at 2026-08-11 12:09:10 on dist-test-slave-nj21"
server_key: "8a3c1f21fb100b75e429f47651366694"
server_key_iv: "5a6d72940351cd6ddee647fbc27b7e12"
server_key_version: "encryptionkey@0"
I20260811 12:09:10.185040 7981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "e7be115158c640c984fb06cdaf10fcb9"
format_stamp: "Formatted at 2026-08-11 12:09:10 on dist-test-slave-nj21"
server_key: "8a3c1f21fb100b75e429f47651366694"
server_key_iv: "5a6d72940351cd6ddee647fbc27b7e12"
server_key_version: "encryptionkey@0"
I20260811 12:09:10.189890 7981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:10.193132 8000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:10.194342 7981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:09:10.194486 7981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "e7be115158c640c984fb06cdaf10fcb9"
format_stamp: "Formatted at 2026-08-11 12:09:10 on dist-test-slave-nj21"
server_key: "8a3c1f21fb100b75e429f47651366694"
server_key_iv: "5a6d72940351cd6ddee647fbc27b7e12"
server_key_version: "encryptionkey@0"
I20260811 12:09:10.194654 7981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:10.215723 7981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:10.219230 7981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:10.219475 7981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:10.222045 7981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:10.222142 7981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:10.222265 7981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:10.222354 7981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:10 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450150, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:10.244427 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.222487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:47033 (local address 127.4.48.254:36117)
0811 12:09:10.222709 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.222716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.224040 (+ 1324us) server_negotiation.cc:407] Connection header received
0811 12:09:10.226747 (+ 2707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.226753 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.226841 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.226962 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:10.229168 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.230351 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.231993 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.232573 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.232699 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.237443 (+ 4744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:10.237448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:10.237451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:10.237483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:10.240134 (+ 2651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.241130 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.241136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.241140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.241308 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.241885 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.241890 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.241893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.242265 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:10.242417 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.243964 (+ 1547us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.244164 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:09:10.245568 8026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.222973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.195:47033)
0811 12:09:10.223855 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.223916 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.226416 (+ 2500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.227292 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.227302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.228136 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:10.228962 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.228992 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.230537 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.230543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.231850 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.231870 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.233072 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.233078 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.233358 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.234001 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:10.234045 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:10.237195 (+ 3150us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:10.240299 (+ 3104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.240308 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.240324 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.240989 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.241431 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.241436 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.241439 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.241724 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.242492 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:10.242505 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:10.242996 (+ 491us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.243349 (+ 353us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.243961 (+ 612us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":336,"threads_started":1}
I20260811 12:09:10.264498 7981 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:46667
I20260811 12:09:10.264500 8234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:46667 every 8 connection(s)
I20260811 12:09:10.266077 7981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:10.268177 8236 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36117
I20260811 12:09:10.268405 8236 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:10.269186 8236 heartbeater.cc:511] Master 127.4.48.254:36117 requested a full tablet report, sending...
I20260811 12:09:10.270243 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7981
I20260811 12:09:10.270416 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:10.270432 8157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.251564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.4.48.195:48405)
0811 12:09:10.256493 (+ 4929us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.256513 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.256671 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.256994 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.256999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.257632 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:10.258088 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.258105 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.259321 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.259330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.260929 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.260946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.262145 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.262155 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.262355 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.263042 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:10.263065 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:10.264232 (+ 1167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:10.267806 (+ 3574us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.267810 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.267813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.268282 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.268698 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.268707 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.268711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.268861 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.269469 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:10.269472 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:10.269622 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.269731 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.270004 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4784,"thread_start_us":4631,"threads_started":1}
I20260811 12:09:10.270458 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.251581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:48405 (local address 127.4.48.254:36117)
0811 12:09:10.251779 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.251786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.256605 (+ 4819us) server_negotiation.cc:407] Connection header received
0811 12:09:10.256793 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.256797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.256855 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.256932 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:10.258327 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.259155 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.261086 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.261735 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.261860 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.264508 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:10.264514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:10.264519 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:10.264555 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:10.267695 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.268416 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.268421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.268425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.268578 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.268992 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.268996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.268999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.269351 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:10.269524 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.269891 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.270110 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:09:10.270699 7378 ts_manager.cc:194] Registered new tserver with Master: e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667)
I20260811 12:09:10.270922 4291 external_mini_cluster.cc:1428] Setting key a016350bd13a215fce03de5c7b1c4cbe
I20260811 12:09:10.271505 7378 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:47033
I20260811 12:09:10.274291 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:10.289189 7737 heartbeater.cc:503] Master 127.4.48.254:36117 was elected leader, sending a full tablet report...
Aug 11 12:09:10 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:09:10 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:10.323515 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.311014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43148 (local address 127.4.48.254:36117)
0811 12:09:10.311177 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.311182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.311529 (+ 347us) server_negotiation.cc:407] Connection header received
0811 12:09:10.311725 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.311729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.311797 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.311889 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:10.313348 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.314072 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.315196 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.315597 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.315715 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.319193 (+ 3478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:10.319199 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:10.319202 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:10.319235 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:10.321484 (+ 2249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.322004 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.322007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.322009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.322124 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.322391 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.322393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.322395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.322645 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:10.322829 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.323020 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.323181 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:09:10.327287 7380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:10.334131 7380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:10.369172 8263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.345536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:46667 (local address 127.0.0.1:42216)
0811 12:09:10.358952 (+ 13416us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.358972 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.359091 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.362580 (+ 3489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.362584 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.362609 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.363128 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.363143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.364890 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.364895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.366316 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.366329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.368225 (+ 1896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.368231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.368420 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.368469 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.368599 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.368789 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13276,"thread_start_us":17399,"threads_started":1}
I20260811 12:09:10.369415 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.345927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42216 (local address 127.4.48.195:46667)
0811 12:09:10.360647 (+ 14720us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.360656 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.360690 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:09:10.360798 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.360808 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.362098 (+ 1290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.362283 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.363326 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.364762 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.366464 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.368025 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.368157 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.368392 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.368921 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.369091 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14527,"thread_start_us":14322,"threads_started":1}
I20260811 12:09:10.369482 8259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.345316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34747 (local address 127.0.0.1:59964)
0811 12:09:10.357951 (+ 12635us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.357988 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.358154 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.361755 (+ 3601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.361764 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.361798 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.362317 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.362330 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.364247 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.364257 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.366586 (+ 2329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.366600 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.368700 (+ 2100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.368707 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.368856 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.368888 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.368983 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.369108 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12391,"mutex_wait_us":2,"thread_start_us":12270,"threads_started":1}
I20260811 12:09:10.370090 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.345519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59964 (local address 127.4.48.193:34747)
0811 12:09:10.360900 (+ 15381us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.360910 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.360940 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:09:10.361042 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.361049 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.361288 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.361483 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.362556 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.364040 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.366771 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.368481 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.368601 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.368796 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.369661 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.369827 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15217,"thread_start_us":15090,"threads_started":1}
I20260811 12:09:10.371380 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.346340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44020 (local address 127.4.48.194:45419)
0811 12:09:10.362759 (+ 16419us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.362767 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.363555 (+ 788us) server_negotiation.cc:407] Connection header received
0811 12:09:10.363778 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.363785 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.363990 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.364171 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.364869 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.366255 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.369251 (+ 2996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.370490 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.370606 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.370789 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.371055 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.371173 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16047,"thread_start_us":16160,"threads_started":1}
I20260811 12:09:10.371425 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.345852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45419 (local address 127.0.0.1:44020)
0811 12:09:10.363437 (+ 17585us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.363455 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.363573 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.364172 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.364177 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.364203 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.364679 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.364689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.366397 (+ 1708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.366402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.369068 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.369082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.370701 (+ 1619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.370709 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.370884 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.370916 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.371016 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.371135 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17466,"thread_start_us":12829,"threads_started":1}
I20260811 12:09:10.373533 8107 tablet_service.cc:1467] Processing CreateTablet for tablet 127cb8775839453291d70b08b92c6664 (DEFAULT_TABLE table=test-table [id=aac38b4b84bc4167a4edae339d70982a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:10.373991 7618 tablet_service.cc:1467] Processing CreateTablet for tablet 127cb8775839453291d70b08b92c6664 (DEFAULT_TABLE table=test-table [id=aac38b4b84bc4167a4edae339d70982a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:10.375038 8107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cb8775839453291d70b08b92c6664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.374634 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 127cb8775839453291d70b08b92c6664 (DEFAULT_TABLE table=test-table [id=aac38b4b84bc4167a4edae339d70982a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:10.375327 7618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cb8775839453291d70b08b92c6664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.375986 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 127cb8775839453291d70b08b92c6664. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.385605 8267 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Bootstrap starting.
I20260811 12:09:10.388664 8267 tablet_bootstrap.cc:654] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.389207 8274 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Bootstrap starting.
I20260811 12:09:10.389971 8275 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Bootstrap starting.
I20260811 12:09:10.390046 8267 log.cc:824] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:10.392690 8274 tablet_bootstrap.cc:654] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.392999 8275 tablet_bootstrap.cc:654] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.393553 8267 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: No bootstrap required, opened a new log
I20260811 12:09:10.393848 8267 ts_tablet_manager.cc:1397] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:09:10.393923 8274 log.cc:824] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:10.394282 8275 log.cc:824] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:10.397225 8275 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: No bootstrap required, opened a new log
I20260811 12:09:10.397259 8274 tablet_bootstrap.cc:492] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: No bootstrap required, opened a new log
I20260811 12:09:10.397490 8274 ts_tablet_manager.cc:1397] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260811 12:09:10.397486 8275 ts_tablet_manager.cc:1397] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:09:10.401677 8267 raft_consensus.cc:348] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.401997 8267 raft_consensus.cc:374] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.402097 8267 raft_consensus.cc:729] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.402626 8267 consensus_queue.cc:260] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.403609 8236 heartbeater.cc:503] Master 127.4.48.254:36117 was elected leader, sending a full tablet report...
I20260811 12:09:10.404256 8267 ts_tablet_manager.cc:1428] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 12:09:10.407732 8275 raft_consensus.cc:348] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.408195 8275 raft_consensus.cc:374] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.408336 8275 raft_consensus.cc:729] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.408185 8274 raft_consensus.cc:348] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.408690 8274 raft_consensus.cc:374] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.408802 8274 raft_consensus.cc:729] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b8d7f3302c496cbb4e445c454d1de0, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.408948 8275 consensus_queue.cc:260] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.409497 8274 consensus_queue.cc:260] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.409948 7974 heartbeater.cc:503] Master 127.4.48.254:36117 was elected leader, sending a full tablet report...
I20260811 12:09:10.410595 8275 ts_tablet_manager.cc:1428] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260811 12:09:10.411052 8274 ts_tablet_manager.cc:1428] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260811 12:09:10.517966 8237 tablet.cc:2369] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:09:10.530442 7975 tablet.cc:2369] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:09:10.533162 7741 tablet.cc:2369] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:10.559398 8284 raft_consensus.cc:482] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:10.559715 8284 raft_consensus.cc:504] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.561408 8284 leader_election.cc:288] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667)
I20260811 12:09:10.567884 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.561957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:36933 (local address 127.4.48.193:34747)
0811 12:09:10.562162 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.562168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.562783 (+ 615us) server_negotiation.cc:407] Connection header received
0811 12:09:10.562976 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.562982 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.563069 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.563144 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.563755 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.564464 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.566221 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.566960 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.567099 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.567175 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.567495 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.567622 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:09:10.567956 8286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.561628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34747 (local address 127.4.48.194:36933)
0811 12:09:10.562660 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.562676 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.562839 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.563179 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.563184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.563208 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.563592 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.563604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.564594 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.564598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.566067 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.566083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.567186 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.567192 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.567319 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.567342 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.567432 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.567548 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":29,"thread_start_us":621,"threads_started":1}
I20260811 12:09:10.567997 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.561837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:50043 (local address 127.4.48.195:46667)
0811 12:09:10.562049 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.562056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.562857 (+ 801us) server_negotiation.cc:407] Connection header received
0811 12:09:10.563056 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.563060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.563119 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.563190 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.563885 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.564601 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.566298 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.567042 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.567194 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.567267 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.567575 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.567725 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:09:10.568512 8287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.561698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:46667 (local address 127.4.48.194:50043)
0811 12:09:10.562769 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.562783 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.562903 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.563216 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.563220 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.563243 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.563732 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.563743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.564732 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.564737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.566154 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.566167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.567246 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.567255 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.567399 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.567421 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.568161 (+ 740us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.568283 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":666,"threads_started":1}
I20260811 12:09:10.569010 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cb8775839453291d70b08b92c6664" candidate_uuid: "96e8f25b2a734d8ea31683ac79882de1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" is_pre_election: true
I20260811 12:09:10.569322 8145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cb8775839453291d70b08b92c6664" candidate_uuid: "96e8f25b2a734d8ea31683ac79882de1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7be115158c640c984fb06cdaf10fcb9" is_pre_election: true
I20260811 12:09:10.569507 7650 raft_consensus.cc:2454] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e8f25b2a734d8ea31683ac79882de1 in term 0.
I20260811 12:09:10.569794 8145 raft_consensus.cc:2454] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96e8f25b2a734d8ea31683ac79882de1 in term 0.
I20260811 12:09:10.570201 7782 leader_election.cc:302] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:10.570606 8284 raft_consensus.cc:2781] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:10.570760 8284 raft_consensus.cc:482] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:10.570858 8284 raft_consensus.cc:3037] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.573030 8284 raft_consensus.cc:504] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.573634 8284 leader_election.cc:288] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [CANDIDATE]: Term 1 election: Requested vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667)
I20260811 12:09:10.574010 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cb8775839453291d70b08b92c6664" candidate_uuid: "96e8f25b2a734d8ea31683ac79882de1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
I20260811 12:09:10.574110 8145 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "127cb8775839453291d70b08b92c6664" candidate_uuid: "96e8f25b2a734d8ea31683ac79882de1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7be115158c640c984fb06cdaf10fcb9"
I20260811 12:09:10.574222 7650 raft_consensus.cc:3037] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.574299 8145 raft_consensus.cc:3037] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.577347 7650 raft_consensus.cc:2454] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e8f25b2a734d8ea31683ac79882de1 in term 1.
I20260811 12:09:10.577350 8145 raft_consensus.cc:2454] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96e8f25b2a734d8ea31683ac79882de1 in term 1.
I20260811 12:09:10.577750 7779 leader_election.cc:302] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, e7be115158c640c984fb06cdaf10fcb9; no voters:
I20260811 12:09:10.578075 8284 raft_consensus.cc:2781] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:10.578477 8284 raft_consensus.cc:686] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 LEADER]: Becoming Leader. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Running, Role: LEADER
I20260811 12:09:10.578904 8284 consensus_queue.cc:237] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } }
I20260811 12:09:10.584488 7380 catalog_manager.cc:5495] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194). New cstate: current_term: 1 leader_uuid: "96e8f25b2a734d8ea31683ac79882de1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } health_report { overall_health: HEALTHY } } }
Aug 11 12:09:10 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450148, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:10.676587 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.662230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36117 (local address 127.0.0.1:43156)
0811 12:09:10.662972 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.662989 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.663106 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.663445 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.663449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.664227 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:10.664529 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.664537 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.665933 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.665936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.666940 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.666952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.667952 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.667956 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.668079 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.668737 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:10.668754 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:10.671155 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:10.673945 (+ 2790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.673957 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.673965 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.674501 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.674884 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:10.674888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:10.674892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:10.675080 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:10.675585 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:10.675594 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:10.675800 (+ 206us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.675992 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.676242 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":433,"threads_started":1}
I20260811 12:09:10.676645 7546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.662185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.4.48.254:36117)
0811 12:09:10.662364 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.662368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.663069 (+ 701us) server_negotiation.cc:407] Connection header received
0811 12:09:10.663204 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.663209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.663312 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.663407 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:10.664679 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.665812 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.667071 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.667779 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.667902 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.671314 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:10.671318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:10.671322 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:10.671363 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:10.673826 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.674600 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.674604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.674606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.674764 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:10.675183 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:10.675186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:10.675188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:10.675473 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:10.675577 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.676076 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.676375 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":47}
I20260811 12:09:10.681123 7380 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:43156:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:09:10.682098 7380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:10.688280 7618 tablet_service.cc:1467] Processing CreateTablet for tablet ad41d4fe39ff428baa38ba90a7ce2f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b0b34612ba4e30a61ec06cd83925c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:10.688627 7855 tablet_service.cc:1467] Processing CreateTablet for tablet ad41d4fe39ff428baa38ba90a7ce2f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b0b34612ba4e30a61ec06cd83925c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:10.688656 8107 tablet_service.cc:1467] Processing CreateTablet for tablet ad41d4fe39ff428baa38ba90a7ce2f43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=71b0b34612ba4e30a61ec06cd83925c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:09:10.689296 7618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad41d4fe39ff428baa38ba90a7ce2f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.689381 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad41d4fe39ff428baa38ba90a7ce2f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.689469 8107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad41d4fe39ff428baa38ba90a7ce2f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:10.695086 8275 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: Bootstrap starting.
I20260811 12:09:10.696347 8274 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: Bootstrap starting.
I20260811 12:09:10.696594 8275 tablet_bootstrap.cc:654] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.696959 8267 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: Bootstrap starting.
I20260811 12:09:10.697976 8274 tablet_bootstrap.cc:654] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.698326 8267 tablet_bootstrap.cc:654] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:10.700304 8274 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: No bootstrap required, opened a new log
I20260811 12:09:10.700455 8274 ts_tablet_manager.cc:1397] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260811 12:09:10.701087 8274 raft_consensus.cc:348] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.701270 8274 raft_consensus.cc:374] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.701370 8274 raft_consensus.cc:729] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b8d7f3302c496cbb4e445c454d1de0, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.701560 8274 consensus_queue.cc:260] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.702138 8274 ts_tablet_manager.cc:1428] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:09:10.701977 8285 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.702380 8285 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.702944 8275 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: No bootstrap required, opened a new log
I20260811 12:09:10.703068 8275 ts_tablet_manager.cc:1397] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260811 12:09:10.703632 8275 raft_consensus.cc:348] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.703903 8275 raft_consensus.cc:374] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.704016 8275 raft_consensus.cc:729] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.704242 8275 consensus_queue.cc:260] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.704722 8275 ts_tablet_manager.cc:1428] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 12:09:10.704910 8284 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.705196 8284 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.705499 8267 tablet_bootstrap.cc:492] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: No bootstrap required, opened a new log
I20260811 12:09:10.705644 8267 ts_tablet_manager.cc:1397] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 12:09:10.706473 8267 raft_consensus.cc:348] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.707038 8267 raft_consensus.cc:374] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:10.707136 8267 raft_consensus.cc:729] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:10.707365 8267 consensus_queue.cc:260] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.707882 8280 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.708243 8280 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.708329 8267 ts_tablet_manager.cc:1428] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:10.864733 8280 raft_consensus.cc:482] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:10.864976 8280 raft_consensus.cc:504] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.866535 8280 leader_election.cc:288] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:10.872759 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.867131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35687 (local address 127.4.48.194:45419)
0811 12:09:10.867304 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.867310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.867757 (+ 447us) server_negotiation.cc:407] Connection header received
0811 12:09:10.867942 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.867945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.868008 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.868081 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.868582 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.869305 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.871090 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.871910 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.872052 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.872127 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.872383 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.872506 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 12:09:10.872867 8305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.866746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45419 (local address 127.4.48.195:35687)
0811 12:09:10.867642 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.867656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.867800 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.868106 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.868110 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.868135 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.868392 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.868402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.869440 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.869455 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.870949 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.870961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.872061 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.872067 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.872233 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.872257 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.872337 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.872469 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":816,"mutex_wait_us":62,"thread_start_us":447,"threads_started":1}
I20260811 12:09:10.873291 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.866895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:34829 (local address 127.4.48.193:34747)
0811 12:09:10.867118 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.867126 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.867665 (+ 539us) server_negotiation.cc:407] Connection header received
0811 12:09:10.867904 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.867908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.867995 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.868082 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:10.868751 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.869550 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.871354 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.872212 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.872376 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.872460 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.872861 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.872990 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:09:10.873371 8307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.866747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34747 (local address 127.4.48.195:34829)
0811 12:09:10.867547 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.867564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.867737 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.868109 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.868114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.868231 (+ 117us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:10.868580 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.868591 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.869700 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.869706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.871178 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.871199 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.872451 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.872457 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.872686 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.872722 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.872831 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.872964 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":21,"spinlock_wait_cycles":25344,"thread_start_us":578,"threads_started":1}
I20260811 12:09:10.874019 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad41d4fe39ff428baa38ba90a7ce2f43" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" is_pre_election: true
I20260811 12:09:10.874248 7650 raft_consensus.cc:2454] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:10.874271 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad41d4fe39ff428baa38ba90a7ce2f43" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1" is_pre_election: true
I20260811 12:09:10.874675 8009 leader_election.cc:302] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7be115158c640c984fb06cdaf10fcb9, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:10.874761 7899 raft_consensus.cc:2454] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:10.875047 8280 raft_consensus.cc:2781] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:10.875165 8280 raft_consensus.cc:482] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:10.875280 8280 raft_consensus.cc:3037] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.877440 8280 raft_consensus.cc:504] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.877933 8280 leader_election.cc:288] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Requested vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:10.878444 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad41d4fe39ff428baa38ba90a7ce2f43" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1"
I20260811 12:09:10.878433 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad41d4fe39ff428baa38ba90a7ce2f43" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
I20260811 12:09:10.878625 7899 raft_consensus.cc:3037] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.878643 7650 raft_consensus.cc:3037] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:10.881529 7650 raft_consensus.cc:2454] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:10.881542 7899 raft_consensus.cc:2454] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:10.881937 8008 leader_election.cc:302] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, e7be115158c640c984fb06cdaf10fcb9; no voters:
I20260811 12:09:10.882251 8280 raft_consensus.cc:2781] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:10.882616 8280 raft_consensus.cc:686] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Becoming Leader. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Running, Role: LEADER
I20260811 12:09:10.883049 8280 consensus_queue.cc:237] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:10.886212 8308 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: TxnStatusTablet state changed. Reason: New leader e7be115158c640c984fb06cdaf10fcb9. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.886531 8308 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:10.887221 8315 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:09:10.887483 8315 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:09:10.888067 7377 catalog_manager.cc:5495] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:10.940929 8298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.935434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:46667 (local address 127.0.0.1:42228)
0811 12:09:10.935727 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.935743 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.935898 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.936359 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.936363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.936376 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:10.936559 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.936567 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.937622 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.937627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.938664 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.938679 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.939363 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.939369 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.939545 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.939598 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:10.940418 (+ 820us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:10.940441 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.940519 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.940664 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 12:09:10.940953 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.935433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42228 (local address 127.4.48.195:46667)
0811 12:09:10.935603 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.935610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.935811 (+ 201us) server_negotiation.cc:407] Connection header received
0811 12:09:10.936088 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.936093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.936189 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.936320 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:10.936681 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.937499 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.938804 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.939197 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.939300 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.939811 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:10.940309 (+ 498us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:10.940446 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.940537 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.940619 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:09:10.948179 7899 raft_consensus.cc:1261] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:10.948405 7650 raft_consensus.cc:1261] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:10.949087 8308 consensus_queue.cc:1035] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:10.949429 8280 consensus_queue.cc:1035] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:10.956550 8285 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: TxnStatusTablet state changed. Reason: New leader e7be115158c640c984fb06cdaf10fcb9. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.956833 8285 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.959532 8308 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.959810 8308 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:10.960410 8321 mvcc.cc:205] Tried to move back new op lower bound from 7317299818274807808 to 7317299818020642816. Current Snapshot: MvccSnapshot[applied={T|T < 7317299818274807808}]
I20260811 12:09:10.962232 8285 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.962462 8285 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P f7b8d7f3302c496cbb4e445c454d1de0: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.962731 8319 mvcc.cc:205] Tried to move back new op lower bound from 7317299818274807808 to 7317299818020642816. Current Snapshot: MvccSnapshot[applied={T|T < 7317299818274807808}]
I20260811 12:09:10.979060 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.971228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44036 (local address 127.4.48.194:45419)
0811 12:09:10.971421 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.971427 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.972080 (+ 653us) server_negotiation.cc:407] Connection header received
0811 12:09:10.972318 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.972324 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.972432 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.972581 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:10.973042 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.974240 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.976237 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.976907 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.977036 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.977519 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:10.978193 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:10.978334 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.978449 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.978536 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:09:10.980533 8284 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: TxnStatusTablet state changed. Reason: New leader e7be115158c640c984fb06cdaf10fcb9. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.980930 8284 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.982015 8308 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.982278 8308 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P e7be115158c640c984fb06cdaf10fcb9: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:09:10.985442 8284 tablet_replica.cc:439] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } } }
I20260811 12:09:10.985689 8284 tablet_replica.cc:442] T ad41d4fe39ff428baa38ba90a7ce2f43 P 96e8f25b2a734d8ea31683ac79882de1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:09:10.999276 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.993144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:52645 (local address 127.4.48.195:46667)
0811 12:09:10.993321 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:10.993326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:10.993887 (+ 561us) server_negotiation.cc:407] Connection header received
0811 12:09:10.994040 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:10.994044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:10.994139 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:10.994206 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:10.994743 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.995443 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.997475 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.997949 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.998040 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.998543 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:10.998640 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:10.998715 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:10.998910 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 12:09:10.998984 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:09:10.999382 8336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:10.993142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:46667 (local address 127.4.48.194:52645)
0811 12:09:10.993781 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:10.993801 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:10.993935 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:10.994242 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:10.994247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:10.994260 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:10.994607 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.994619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.995564 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.995570 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.997325 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:10.997341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:10.998110 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:10.998117 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:10.998293 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:10.998362 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:10.998783 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:10.998793 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 12:09:10.998861 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 12:09:10.998986 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":347,"threads_started":1}
I20260811 12:09:11.008796 7650 raft_consensus.cc:1261] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Refusing update from remote peer 96e8f25b2a734d8ea31683ac79882de1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:11.008935 8145 raft_consensus.cc:1261] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Refusing update from remote peer 96e8f25b2a734d8ea31683ac79882de1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:11.009572 8289 consensus_queue.cc:1035] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:11.009997 8284 consensus_queue.cc:1035] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:11.020231 8322 mvcc.cc:205] Tried to move back new op lower bound from 7317299818524971008 to 7317299816774873088. Current Snapshot: MvccSnapshot[applied={T|T < 7317299818524971008}]
I20260811 12:09:11.049118 8265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:11.040197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:47207 (local address 127.4.48.194:45419)
0811 12:09:11.040437 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:11.040443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:11.043270 (+ 2827us) server_negotiation.cc:407] Connection header received
0811 12:09:11.043448 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:11.043454 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:11.043530 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:11.043600 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:11.044492 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.045597 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.047299 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.047876 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.047983 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:11.048476 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:11.048571 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:11.048643 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:11.048775 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 12:09:11.048854 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260811 12:09:11.049299 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:11.040231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45419 (local address 127.4.48.195:47207)
0811 12:09:11.043173 (+ 2942us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:11.043189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:11.043321 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:11.043635 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:11.043640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:11.043652 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:11.044337 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:11.044351 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.045729 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.045732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:11.047139 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:11.047153 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.048049 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.048056 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:11.048227 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:11.048300 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:11.048663 (+ 363us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:11.048674 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:09:11.048775 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 12:09:11.048915 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2762,"thread_start_us":2597,"threads_started":1}
I20260811 12:09:11.079756 7377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:11.080044 7377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:11.083180 7377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Sending DeleteTablet for 3 replicas of tablet 127cb8775839453291d70b08b92c6664
I20260811 12:09:11.084371 8107 tablet_service.cc:1514] Processing DeleteTablet for tablet 127cb8775839453291d70b08b92c6664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42216
I20260811 12:09:11.084451 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 127cb8775839453291d70b08b92c6664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:44020
I20260811 12:09:11.085381 7618 tablet_service.cc:1514] Processing DeleteTablet for tablet 127cb8775839453291d70b08b92c6664 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:59964
I20260811 12:09:11.086268 7377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:11.086887 7377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:11.094730 7614 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf08709cac44f198cb12d3135201295 (DEFAULT_TABLE table=test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:11.095304 7614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf08709cac44f198cb12d3135201295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:11.098553 7853 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf08709cac44f198cb12d3135201295 (DEFAULT_TABLE table=test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:11.099110 7853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf08709cac44f198cb12d3135201295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:11.104404 8102 tablet_service.cc:1467] Processing CreateTablet for tablet 5cf08709cac44f198cb12d3135201295 (DEFAULT_TABLE table=test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:11.104966 8102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5cf08709cac44f198cb12d3135201295. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:11.110148 8359 tablet_replica.cc:330] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: stopping tablet replica
I20260811 12:09:11.110744 8359 raft_consensus.cc:2229] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:11.111758 8267 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Bootstrap starting.
I20260811 12:09:11.113739 8267 tablet_bootstrap.cc:654] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:11.115662 8275 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Bootstrap starting.
I20260811 12:09:11.117416 8275 tablet_bootstrap.cc:654] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:11.120246 8358 tablet_replica.cc:330] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: stopping tablet replica
I20260811 12:09:11.120893 8358 raft_consensus.cc:2229] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:11.121189 8274 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Bootstrap starting.
I20260811 12:09:11.121371 8359 raft_consensus.cc:2258] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:11.122881 8274 tablet_bootstrap.cc:654] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:11.123334 8358 raft_consensus.cc:2258] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:11.125069 8359 ts_tablet_manager.cc:1905] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:11.126793 8357 tablet_replica.cc:330] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: stopping tablet replica
I20260811 12:09:11.130317 8357 raft_consensus.cc:2229] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:11.132022 8358 ts_tablet_manager.cc:1905] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:11.132915 8267 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: No bootstrap required, opened a new log
I20260811 12:09:11.133219 8267 ts_tablet_manager.cc:1397] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Time spent bootstrapping tablet: real 0.022s user 0.003s sys 0.007s
I20260811 12:09:11.134121 8267 raft_consensus.cc:348] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.134528 8267 raft_consensus.cc:374] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:11.134658 8267 raft_consensus.cc:729] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:11.135027 8267 consensus_queue.cc:260] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.136046 8274 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: No bootstrap required, opened a new log
I20260811 12:09:11.136219 8274 ts_tablet_manager.cc:1397] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260811 12:09:11.138016 8274 raft_consensus.cc:348] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.138264 8274 raft_consensus.cc:374] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:11.138352 8274 raft_consensus.cc:729] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b8d7f3302c496cbb4e445c454d1de0, State: Initialized, Role: FOLLOWER
I20260811 12:09:11.138561 8267 ts_tablet_manager.cc:1428] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s
I20260811 12:09:11.138720 8274 consensus_queue.cc:260] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.140479 8274 ts_tablet_manager.cc:1428] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:09:11.147509 8358 ts_tablet_manager.cc:1918] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:11.138674 8275 tablet_bootstrap.cc:492] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: No bootstrap required, opened a new log
I20260811 12:09:11.147928 8275 ts_tablet_manager.cc:1397] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Time spent bootstrapping tablet: real 0.032s user 0.013s sys 0.000s
I20260811 12:09:11.148386 8357 raft_consensus.cc:2258] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:11.148913 8275 raft_consensus.cc:348] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.149195 8275 raft_consensus.cc:374] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:11.149276 8275 raft_consensus.cc:729] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Initialized, Role: FOLLOWER
I20260811 12:09:11.149531 8275 consensus_queue.cc:260] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.150138 8275 ts_tablet_manager.cc:1428] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:09:11.153980 8359 ts_tablet_manager.cc:1918] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:11.154138 8359 log.cc:1196] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/127cb8775839453291d70b08b92c6664
I20260811 12:09:11.154706 8359 ts_tablet_manager.cc:1939] T 127cb8775839453291d70b08b92c6664 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting consensus metadata
I20260811 12:09:11.155977 8357 ts_tablet_manager.cc:1905] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:11.156466 8358 log.cc:1196] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/127cb8775839453291d70b08b92c6664
I20260811 12:09:11.156625 7356 catalog_manager.cc:4841] TS f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747): tablet 127cb8775839453291d70b08b92c6664 (table test-table [id=aac38b4b84bc4167a4edae339d70982a]) successfully deleted
I20260811 12:09:11.157071 8358 ts_tablet_manager.cc:1939] T 127cb8775839453291d70b08b92c6664 P e7be115158c640c984fb06cdaf10fcb9: Deleting consensus metadata
I20260811 12:09:11.158716 7352 catalog_manager.cc:4841] TS e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667): tablet 127cb8775839453291d70b08b92c6664 (table test-table [id=aac38b4b84bc4167a4edae339d70982a]) successfully deleted
I20260811 12:09:11.163398 8357 ts_tablet_manager.cc:1918] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:11.164045 8357 log.cc:1196] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/127cb8775839453291d70b08b92c6664
I20260811 12:09:11.164474 8357 ts_tablet_manager.cc:1939] T 127cb8775839453291d70b08b92c6664 P 96e8f25b2a734d8ea31683ac79882de1: Deleting consensus metadata
I20260811 12:09:11.165712 7354 catalog_manager.cc:4841] TS 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419): tablet 127cb8775839453291d70b08b92c6664 (table test-table [id=aac38b4b84bc4167a4edae339d70982a]) successfully deleted
I20260811 12:09:11.283857 8308 raft_consensus.cc:482] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:11.284060 8308 raft_consensus.cc:504] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.284796 8308 leader_election.cc:288] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:11.285806 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf08709cac44f198cb12d3135201295" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" is_pre_election: true
I20260811 12:09:11.286044 7650 raft_consensus.cc:2454] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:11.286775 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf08709cac44f198cb12d3135201295" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1" is_pre_election: true
I20260811 12:09:11.287020 7899 raft_consensus.cc:2454] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:11.287595 8009 leader_election.cc:302] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7be115158c640c984fb06cdaf10fcb9, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:11.288094 8308 raft_consensus.cc:2781] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:11.288194 8308 raft_consensus.cc:482] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:11.288305 8308 raft_consensus.cc:3037] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:11.291471 8308 raft_consensus.cc:504] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.292474 7650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf08709cac44f198cb12d3135201295" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
I20260811 12:09:11.292655 7650 raft_consensus.cc:3037] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:11.295493 7650 raft_consensus.cc:2454] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:11.295931 8009 leader_election.cc:302] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7be115158c640c984fb06cdaf10fcb9, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:11.296228 8337 raft_consensus.cc:2781] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:11.296398 8337 raft_consensus.cc:686] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Becoming Leader. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Running, Role: LEADER
I20260811 12:09:11.296756 8337 consensus_queue.cc:237] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:11.297509 8308 leader_election.cc:288] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Requested vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:11.298398 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5cf08709cac44f198cb12d3135201295" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1"
I20260811 12:09:11.298604 7899 raft_consensus.cc:3037] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:11.301666 7899 raft_consensus.cc:2454] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:11.314607 7377 catalog_manager.cc:5495] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:11.371654 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:11.365200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42232 (local address 127.4.48.195:46667)
0811 12:09:11.365361 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:11.365366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:11.365382 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:11.365435 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:11.365440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:11.365511 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:11.365582 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:11.366108 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.367223 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.369092 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.369809 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.369926 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:11.370478 (+ 552us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:11.370626 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:11.370698 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:11.371260 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 12:09:11.371340 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:09:11.389947 8264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:11.384034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:54177 (local address 127.4.48.195:46667)
0811 12:09:11.384180 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:11.384186 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:11.384203 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:09:11.384255 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:11.384260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:11.384333 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:11.384423 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:11.384791 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.385918 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.387954 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.388597 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.388735 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:11.389179 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:11.389308 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:11.389401 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:11.389538 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 12:09:11.389621 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":46}
I20260811 12:09:11.389989 8351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:11.383790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:46667 (local address 127.4.48.195:54177)
0811 12:09:11.383974 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:11.383992 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:11.384090 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:11.384439 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:11.384444 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:11.384455 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:09:11.384666 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:11.384675 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.386067 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.386073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:11.387807 (+ 1734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:11.387820 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:11.388750 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:11.388756 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:11.388966 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:11.389013 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:09:11.389410 (+ 397us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:09:11.389422 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:09:11.389509 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:09:11.389652 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":23}
I20260811 12:09:11.401083 7650 raft_consensus.cc:1261] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:11.401149 7899 raft_consensus.cc:1261] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:11.401639 8280 consensus_queue.cc:1035] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:11.401873 8337 consensus_queue.cc:1035] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:09:11.433152 8319 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:09:13.030215 7377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:13.030479 7377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:13.034430 7377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Sending DeleteTablet for 3 replicas of tablet 5cf08709cac44f198cb12d3135201295
I20260811 12:09:13.035248 7614 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf08709cac44f198cb12d3135201295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:59964
I20260811 12:09:13.035307 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf08709cac44f198cb12d3135201295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:44020
I20260811 12:09:13.035513 8107 tablet_service.cc:1514] Processing DeleteTablet for tablet 5cf08709cac44f198cb12d3135201295 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42216
I20260811 12:09:13.047834 8413 tablet_replica.cc:330] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: stopping tablet replica
I20260811 12:09:13.048121 8410 tablet_replica.cc:330] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: stopping tablet replica
I20260811 12:09:13.048364 8410 raft_consensus.cc:2229] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:13.048425 8411 tablet_replica.cc:330] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: stopping tablet replica
I20260811 12:09:13.048452 8413 raft_consensus.cc:2229] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:13.048625 8411 raft_consensus.cc:2229] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:13.048713 8410 raft_consensus.cc:2258] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:13.049212 8413 raft_consensus.cc:2258] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:13.048920 8411 raft_consensus.cc:2258] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:13.049779 8410 ts_tablet_manager.cc:1905] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.050302 8411 ts_tablet_manager.cc:1905] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.050391 8413 ts_tablet_manager.cc:1905] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.056643 8411 ts_tablet_manager.cc:1918] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:13.056843 8411 log.cc:1196] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/5cf08709cac44f198cb12d3135201295
I20260811 12:09:13.057143 8410 ts_tablet_manager.cc:1918] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:13.057132 8413 ts_tablet_manager.cc:1918] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:13.057291 8413 log.cc:1196] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/5cf08709cac44f198cb12d3135201295
I20260811 12:09:13.057302 8411 ts_tablet_manager.cc:1939] T 5cf08709cac44f198cb12d3135201295 P 96e8f25b2a734d8ea31683ac79882de1: Deleting consensus metadata
I20260811 12:09:13.057399 8410 log.cc:1196] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/5cf08709cac44f198cb12d3135201295
I20260811 12:09:13.057885 8413 ts_tablet_manager.cc:1939] T 5cf08709cac44f198cb12d3135201295 P e7be115158c640c984fb06cdaf10fcb9: Deleting consensus metadata
I20260811 12:09:13.057914 8410 ts_tablet_manager.cc:1939] T 5cf08709cac44f198cb12d3135201295 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting consensus metadata
I20260811 12:09:13.058358 7354 catalog_manager.cc:4841] TS 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419): tablet 5cf08709cac44f198cb12d3135201295 (table test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]) successfully deleted
I20260811 12:09:13.059041 7356 catalog_manager.cc:4841] TS f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747): tablet 5cf08709cac44f198cb12d3135201295 (table test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]) successfully deleted
I20260811 12:09:13.059476 7352 catalog_manager.cc:4841] TS e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667): tablet 5cf08709cac44f198cb12d3135201295 (table test-table [id=142fd6a4f1744bd588a42378d0a1a6c4]) successfully deleted
Aug 11 12:09:13 dist-test-slave-nj21 krb5kdc[7293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:09:13 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:13.096089 8422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:13.081962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43158 (local address 127.4.48.254:36117)
0811 12:09:13.082542 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:13.082548 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:13.082616 (+ 68us) server_negotiation.cc:407] Connection header received
0811 12:09:13.082813 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:13.082819 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:13.082889 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:13.082982 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:13.084331 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:13.085126 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:13.086365 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:13.086789 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:13.086901 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:13.090748 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:13.090753 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:13.090756 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:13.090791 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:13.093440 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:13.094071 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:13.094077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:13.094080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:13.094218 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:13.094553 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:13.094556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:13.094558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:13.094921 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:13.095196 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:13.095465 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 12:09:13.095720 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":296,"threads_started":1}
I20260811 12:09:13.099328 7377 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:13.099958 7377 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:13.106196 7614 tablet_service.cc:1467] Processing CreateTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 (DEFAULT_TABLE table=test-table [id=f9c52f8f84854df38da3fd635dcf8c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.106254 7855 tablet_service.cc:1467] Processing CreateTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 (DEFAULT_TABLE table=test-table [id=f9c52f8f84854df38da3fd635dcf8c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.106283 8107 tablet_service.cc:1467] Processing CreateTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 (DEFAULT_TABLE table=test-table [id=f9c52f8f84854df38da3fd635dcf8c16]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.106671 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ddb1c8fd31470e9eae577f3f7fa518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.106673 8107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ddb1c8fd31470e9eae577f3f7fa518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.106694 7614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91ddb1c8fd31470e9eae577f3f7fa518. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.113577 8425 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Bootstrap starting.
I20260811 12:09:13.113811 8424 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Bootstrap starting.
I20260811 12:09:13.114281 8426 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Bootstrap starting.
I20260811 12:09:13.115571 8425 tablet_bootstrap.cc:654] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.115649 8424 tablet_bootstrap.cc:654] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.115906 8426 tablet_bootstrap.cc:654] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.119511 8426 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: No bootstrap required, opened a new log
I20260811 12:09:13.119639 8426 ts_tablet_manager.cc:1397] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 12:09:13.120074 8424 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: No bootstrap required, opened a new log
I20260811 12:09:13.120232 8424 ts_tablet_manager.cc:1397] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:09:13.120440 8426 raft_consensus.cc:348] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.120626 8426 raft_consensus.cc:374] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.120714 8426 raft_consensus.cc:729] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b8d7f3302c496cbb4e445c454d1de0, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.120746 8425 tablet_bootstrap.cc:492] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: No bootstrap required, opened a new log
I20260811 12:09:13.120898 8425 ts_tablet_manager.cc:1397] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 12:09:13.120970 8426 consensus_queue.cc:260] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.121006 8424 raft_consensus.cc:348] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.121259 8424 raft_consensus.cc:374] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.121342 8424 raft_consensus.cc:729] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.121658 8424 consensus_queue.cc:260] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.121711 8425 raft_consensus.cc:348] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.121968 8425 raft_consensus.cc:374] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.122092 8425 raft_consensus.cc:729] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.122354 8425 consensus_queue.cc:260] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.123031 8425 ts_tablet_manager.cc:1428] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:09:13.128317 8426 ts_tablet_manager.cc:1428] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260811 12:09:13.128727 8424 ts_tablet_manager.cc:1428] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Time spent starting tablet: real 0.008s user 0.005s sys 0.004s
I20260811 12:09:13.182662 8372 raft_consensus.cc:482] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:13.182857 8372 raft_consensus.cc:504] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.183528 8372 leader_election.cc:288] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:13.184016 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ddb1c8fd31470e9eae577f3f7fa518" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" is_pre_election: true
I20260811 12:09:13.184150 7896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ddb1c8fd31470e9eae577f3f7fa518" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1" is_pre_election: true
I20260811 12:09:13.184270 7648 raft_consensus.cc:2454] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:13.184412 7896 raft_consensus.cc:2454] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:13.184742 8009 leader_election.cc:302] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7be115158c640c984fb06cdaf10fcb9, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:13.185010 8372 raft_consensus.cc:2781] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:13.185134 8372 raft_consensus.cc:482] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:13.185223 8372 raft_consensus.cc:3037] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.187495 8372 raft_consensus.cc:504] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.188017 8372 leader_election.cc:288] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Requested vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:13.188423 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ddb1c8fd31470e9eae577f3f7fa518" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
I20260811 12:09:13.188522 7896 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91ddb1c8fd31470e9eae577f3f7fa518" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1"
I20260811 12:09:13.188660 7648 raft_consensus.cc:3037] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.188706 7896 raft_consensus.cc:3037] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.191586 7896 raft_consensus.cc:2454] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:13.191658 7648 raft_consensus.cc:2454] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:13.192019 8008 leader_election.cc:302] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, e7be115158c640c984fb06cdaf10fcb9, f7b8d7f3302c496cbb4e445c454d1de0; no voters:
I20260811 12:09:13.192319 8372 raft_consensus.cc:2781] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:13.192468 8372 raft_consensus.cc:686] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Becoming Leader. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Running, Role: LEADER
I20260811 12:09:13.192727 8372 consensus_queue.cc:237] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.195209 7380 catalog_manager.cc:5495] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:13.222513 8444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:13.217199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42248 (local address 127.4.48.195:46667)
0811 12:09:13.217899 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:13.217904 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:13.217926 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:09:13.217973 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:13.217977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:13.218032 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:13.218122 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:13.218569 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:13.219399 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:13.220774 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:13.221210 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:13.221320 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:13.221765 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:13.221882 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:13.221961 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:13.222109 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:09:13.222185 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":389,"threads_started":1}
I20260811 12:09:13.230216 7899 raft_consensus.cc:1261] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:13.230316 7648 raft_consensus.cc:1261] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:13.230702 8372 consensus_queue.cc:1035] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:13.230969 8376 consensus_queue.cc:1035] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:13.273605 7377 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:13.273849 7377 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:13.276999 7377 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Sending DeleteTablet for 3 replicas of tablet 91ddb1c8fd31470e9eae577f3f7fa518
I20260811 12:09:13.277880 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:44020
I20260811 12:09:13.277891 8102 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42216
I20260811 12:09:13.278139 8411 tablet_replica.cc:330] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: stopping tablet replica
I20260811 12:09:13.278152 8413 tablet_replica.cc:330] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: stopping tablet replica
I20260811 12:09:13.278324 8411 raft_consensus.cc:2229] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:13.278342 8413 raft_consensus.cc:2229] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:13.278496 8411 raft_consensus.cc:2258] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:13.278648 8413 raft_consensus.cc:2258] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:13.279881 8413 ts_tablet_manager.cc:1905] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.284101 7614 tablet_service.cc:1514] Processing DeleteTablet for tablet 91ddb1c8fd31470e9eae577f3f7fa518 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:13 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:59964
I20260811 12:09:13.284363 8410 tablet_replica.cc:330] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: stopping tablet replica
I20260811 12:09:13.284271 7380 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 12:09:13.284543 8410 raft_consensus.cc:2229] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:13.284669 8411 ts_tablet_manager.cc:1905] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.284725 8410 raft_consensus.cc:2258] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 12:09:13.284965 7380 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:13.286091 8410 ts_tablet_manager.cc:1905] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:13.289285 8413 ts_tablet_manager.cc:1918] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:13.289544 8413 log.cc:1196] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/91ddb1c8fd31470e9eae577f3f7fa518
I20260811 12:09:13.291294 8411 ts_tablet_manager.cc:1918] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:13.291908 8413 ts_tablet_manager.cc:1939] T 91ddb1c8fd31470e9eae577f3f7fa518 P e7be115158c640c984fb06cdaf10fcb9: Deleting consensus metadata
I20260811 12:09:13.292312 8410 ts_tablet_manager.cc:1918] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:09:13.292552 8410 log.cc:1196] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/91ddb1c8fd31470e9eae577f3f7fa518
I20260811 12:09:13.293604 7614 tablet_service.cc:1467] Processing CreateTablet for tablet d0010accbd2c49219798923ca9e00d09 (DEFAULT_TABLE table=test-table [id=3562978a400048628dcdbe854d349e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.294036 7614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0010accbd2c49219798923ca9e00d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.294013 8102 tablet_service.cc:1467] Processing CreateTablet for tablet d0010accbd2c49219798923ca9e00d09 (DEFAULT_TABLE table=test-table [id=3562978a400048628dcdbe854d349e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.294430 8102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0010accbd2c49219798923ca9e00d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.298184 8411 log.cc:1196] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/91ddb1c8fd31470e9eae577f3f7fa518
I20260811 12:09:13.298620 8411 ts_tablet_manager.cc:1939] T 91ddb1c8fd31470e9eae577f3f7fa518 P 96e8f25b2a734d8ea31683ac79882de1: Deleting consensus metadata
I20260811 12:09:13.299574 7855 tablet_service.cc:1467] Processing CreateTablet for tablet d0010accbd2c49219798923ca9e00d09 (DEFAULT_TABLE table=test-table [id=3562978a400048628dcdbe854d349e26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:13.299965 7855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0010accbd2c49219798923ca9e00d09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:13.300668 7352 catalog_manager.cc:4841] TS e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667): tablet 91ddb1c8fd31470e9eae577f3f7fa518 (table test-table [id=f9c52f8f84854df38da3fd635dcf8c16]) successfully deleted
I20260811 12:09:13.303249 8425 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Bootstrap starting.
I20260811 12:09:13.303763 7354 catalog_manager.cc:4841] TS 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419): tablet 91ddb1c8fd31470e9eae577f3f7fa518 (table test-table [id=f9c52f8f84854df38da3fd635dcf8c16]) successfully deleted
I20260811 12:09:13.304999 8425 tablet_bootstrap.cc:654] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.305660 8426 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Bootstrap starting.
I20260811 12:09:13.306272 8424 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Bootstrap starting.
I20260811 12:09:13.306516 8410 ts_tablet_manager.cc:1939] T 91ddb1c8fd31470e9eae577f3f7fa518 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting consensus metadata
I20260811 12:09:13.307495 8426 tablet_bootstrap.cc:654] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.307662 7356 catalog_manager.cc:4841] TS f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747): tablet 91ddb1c8fd31470e9eae577f3f7fa518 (table test-table [id=f9c52f8f84854df38da3fd635dcf8c16]) successfully deleted
I20260811 12:09:13.307912 8424 tablet_bootstrap.cc:654] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:13.310426 8425 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: No bootstrap required, opened a new log
I20260811 12:09:13.310597 8425 ts_tablet_manager.cc:1397] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.005s
I20260811 12:09:13.311290 8426 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: No bootstrap required, opened a new log
I20260811 12:09:13.311532 8426 ts_tablet_manager.cc:1397] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 12:09:13.311404 8425 raft_consensus.cc:348] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.311853 8425 raft_consensus.cc:374] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.312022 8425 raft_consensus.cc:729] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.312316 8425 consensus_queue.cc:260] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.313452 8425 ts_tablet_manager.cc:1428] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 12:09:13.314085 8426 raft_consensus.cc:348] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.314379 8426 raft_consensus.cc:374] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.314494 8426 raft_consensus.cc:729] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7b8d7f3302c496cbb4e445c454d1de0, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.314783 8424 tablet_bootstrap.cc:492] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: No bootstrap required, opened a new log
I20260811 12:09:13.314749 8426 consensus_queue.cc:260] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.314960 8424 ts_tablet_manager.cc:1397] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260811 12:09:13.315347 8426 ts_tablet_manager.cc:1428] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s
I20260811 12:09:13.316502 8424 raft_consensus.cc:348] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.316790 8424 raft_consensus.cc:374] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:13.316870 8424 raft_consensus.cc:729] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96e8f25b2a734d8ea31683ac79882de1, State: Initialized, Role: FOLLOWER
I20260811 12:09:13.317121 8424 consensus_queue.cc:260] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.317684 8424 ts_tablet_manager.cc:1428] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Time spent starting tablet: real 0.003s user 0.000s sys 0.000s
I20260811 12:09:13.359066 8396 raft_consensus.cc:482] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:13.359270 8396 raft_consensus.cc:504] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.359908 8396 leader_election.cc:288] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:13.360456 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0010accbd2c49219798923ca9e00d09" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1" is_pre_election: true
I20260811 12:09:13.360524 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0010accbd2c49219798923ca9e00d09" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" is_pre_election: true
I20260811 12:09:13.360707 7899 raft_consensus.cc:2454] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:13.360759 7648 raft_consensus.cc:2454] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 0.
I20260811 12:09:13.361119 8008 leader_election.cc:302] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, e7be115158c640c984fb06cdaf10fcb9; no voters:
I20260811 12:09:13.361389 8396 raft_consensus.cc:2781] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:13.361495 8396 raft_consensus.cc:482] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:13.361562 8396 raft_consensus.cc:3037] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.363977 8396 raft_consensus.cc:504] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.364516 8396 leader_election.cc:288] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Requested vote from peers f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747), 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419)
I20260811 12:09:13.364944 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0010accbd2c49219798923ca9e00d09" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7b8d7f3302c496cbb4e445c454d1de0"
I20260811 12:09:13.364969 7899 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0010accbd2c49219798923ca9e00d09" candidate_uuid: "e7be115158c640c984fb06cdaf10fcb9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96e8f25b2a734d8ea31683ac79882de1"
I20260811 12:09:13.365126 7648 raft_consensus.cc:3037] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.365175 7899 raft_consensus.cc:3037] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:13.367161 7899 raft_consensus.cc:2454] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:13.367267 7648 raft_consensus.cc:2454] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e7be115158c640c984fb06cdaf10fcb9 in term 1.
I20260811 12:09:13.367520 8008 leader_election.cc:302] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 96e8f25b2a734d8ea31683ac79882de1, e7be115158c640c984fb06cdaf10fcb9; no voters:
I20260811 12:09:13.367880 8396 raft_consensus.cc:2781] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:13.368003 8396 raft_consensus.cc:686] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Becoming Leader. State: Replica: e7be115158c640c984fb06cdaf10fcb9, State: Running, Role: LEADER
I20260811 12:09:13.368307 8396 consensus_queue.cc:237] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } }
I20260811 12:09:13.370658 7380 catalog_manager.cc:5495] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "e7be115158c640c984fb06cdaf10fcb9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e7be115158c640c984fb06cdaf10fcb9" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 46667 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:13.395607 7648 raft_consensus.cc:1261] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:13.395890 7899 raft_consensus.cc:1261] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Refusing update from remote peer e7be115158c640c984fb06cdaf10fcb9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:13.396237 8372 consensus_queue.cc:1035] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7b8d7f3302c496cbb4e445c454d1de0" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:13.396500 8396 consensus_queue.cc:1035] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "96e8f25b2a734d8ea31683ac79882de1" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:09:13.415205 8373 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 12:09:14.995244 7380 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:14.995467 7380 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:14.999518 7380 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0162b84de83c4e4380d6632a1c8e224c: Sending DeleteTablet for 3 replicas of tablet d0010accbd2c49219798923ca9e00d09
I20260811 12:09:15.000370 7614 tablet_service.cc:1514] Processing DeleteTablet for tablet d0010accbd2c49219798923ca9e00d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:14 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:59964
I20260811 12:09:15.000476 8107 tablet_service.cc:1514] Processing DeleteTablet for tablet d0010accbd2c49219798923ca9e00d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:14 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42216
I20260811 12:09:15.000548 7855 tablet_service.cc:1514] Processing DeleteTablet for tablet d0010accbd2c49219798923ca9e00d09 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:14 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:44020
I20260811 12:09:15.001132 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7500
I20260811 12:09:15.013461 8500 tablet_replica.cc:330] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: stopping tablet replica
I20260811 12:09:15.014212 8500 raft_consensus.cc:2229] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:15.014573 8500 raft_consensus.cc:2258] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:15.015297 8498 tablet_replica.cc:330] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: stopping tablet replica
I20260811 12:09:15.015614 8498 raft_consensus.cc:2229] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:15.015784 8500 ts_tablet_manager.cc:1905] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:15.015980 8498 raft_consensus.cc:2258] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:15.017133 8498 ts_tablet_manager.cc:1905] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:15.022791 8499 tablet_replica.cc:330] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: stopping tablet replica
I20260811 12:09:15.022987 8499 raft_consensus.cc:2229] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:15.023278 8499 raft_consensus.cc:2258] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:15.024194 8500 ts_tablet_manager.cc:1918] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:15.024341 8500 log.cc:1196] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/d0010accbd2c49219798923ca9e00d09
I20260811 12:09:15.024336 8499 ts_tablet_manager.cc:1905] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:15.024386 8498 ts_tablet_manager.cc:1918] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:15.024572 8498 log.cc:1196] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/d0010accbd2c49219798923ca9e00d09
I20260811 12:09:15.024837 8500 ts_tablet_manager.cc:1939] T d0010accbd2c49219798923ca9e00d09 P e7be115158c640c984fb06cdaf10fcb9: Deleting consensus metadata
I20260811 12:09:15.024991 8498 ts_tablet_manager.cc:1939] T d0010accbd2c49219798923ca9e00d09 P f7b8d7f3302c496cbb4e445c454d1de0: Deleting consensus metadata
I20260811 12:09:15.025966 7352 catalog_manager.cc:4841] TS e7be115158c640c984fb06cdaf10fcb9 (127.4.48.195:46667): tablet d0010accbd2c49219798923ca9e00d09 (table test-table [id=3562978a400048628dcdbe854d349e26]) successfully deleted
I20260811 12:09:15.025979 7356 catalog_manager.cc:4841] TS f7b8d7f3302c496cbb4e445c454d1de0 (127.4.48.193:34747): tablet d0010accbd2c49219798923ca9e00d09 (table test-table [id=3562978a400048628dcdbe854d349e26]) successfully deleted
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:15.030193 8499 ts_tablet_manager.cc:1918] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 12:09:15.030304 8499 log.cc:1196] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/d0010accbd2c49219798923ca9e00d09
I20260811 12:09:15.030723 8499 ts_tablet_manager.cc:1939] T d0010accbd2c49219798923ca9e00d09 P 96e8f25b2a734d8ea31683ac79882de1: Deleting consensus metadata
I20260811 12:09:15.031600 7354 catalog_manager.cc:4841] TS 96e8f25b2a734d8ea31683ac79882de1 (127.4.48.194:45419): tablet d0010accbd2c49219798923ca9e00d09 (table test-table [id=3562978a400048628dcdbe854d349e26]) successfully deleted
I20260811 12:09:15.035581 8502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:15.002587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40698 (local address 127.4.48.193:34747)
0811 12:09:15.014393 (+ 11806us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:15.014400 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:15.014424 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:09:15.014486 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:15.014492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:15.014573 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:15.014690 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:15.016848 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.017973 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.019899 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.021113 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.021262 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:15.029901 (+ 8639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:15.029917 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:15.029937 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:15.029975 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:15.032971 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.033701 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.033710 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.033712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.033876 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.034218 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.034221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.034222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.034564 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:15.034780 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:15.035052 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:09:15.035217 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11641,"thread_start_us":11492,"threads_started":1}
W20260811 12:09:15.036286 7729 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40698
W20260811 12:09:15.036870 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7500: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:15.037058 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7500
I20260811 12:09:15.064517 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7748
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:15.078959 8503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:15.065214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.4.48.194:45419)
0811 12:09:15.065733 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:15.065737 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:15.065752 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:09:15.065803 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:15.065806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:15.065875 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:15.065977 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:15.067191 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.067971 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.069199 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.069607 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.069717 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:15.073736 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:15.073749 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:15.073765 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:15.073796 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:15.076382 (+ 2586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.076988 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.076995 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.076998 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.077122 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.077467 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.077471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.077473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.077819 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:15.078130 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:15.078390 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 12:09:15.078635 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":259,"threads_started":1}
W20260811 12:09:15.079576 7969 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39862
W20260811 12:09:15.080255 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:15.080389 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7748
I20260811 12:09:15.108773 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7981
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[7293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:15.122870 8504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:15.109584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.4.48.195:46667)
0811 12:09:15.110084 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:15.110088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:15.110104 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:15.110156 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:15.110160 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:15.110224 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:15.110311 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:15.111499 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.112251 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.113393 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:15.113793 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:15.113879 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:15.117929 (+ 4050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:15.117945 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:15.117964 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:15.117996 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:15.120689 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.121263 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.121269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.121272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.121391 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:15.121658 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:15.121661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:15.121663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:15.122010 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:15.122229 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:15.122422 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:09:15.122601 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":246,"threads_started":1}
W20260811 12:09:15.123531 8225 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60184
W20260811 12:09:15.124228 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7981: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:15.124396 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7981
I20260811 12:09:15.158744 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7315
I20260811 12:09:15.159968 7444 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:15.275436 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7315
2026-08-11T12:09:15Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8425 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[8509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[8509](info): set up 2 sockets
Aug 11 12:09:15 dist-test-slave-nj21 krb5kdc[8509](info): commencing operation
krb5kdc: starting...
W20260811 12:09:16.491852 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:16 dist-test-slave-nj21 krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:16.681195 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43221
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:43221
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:17.004093 8526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:17.004379 8526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:17.004519 8526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:17.011451 8526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:17.011556 8526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:17.011638 8526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:17.011729 8526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:17.011799 8526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:17.021119 8526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:43221
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:17.022339 8526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:17.023634 8526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:17.035264 8534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:17.035203 8540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:17.035235 8535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:17.035727 8526 server_base.cc:1034] running on GCE node
I20260811 12:09:17.036608 8526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:17.037897 8526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:17.039099 8526 hybrid_clock.cc:648] HybridClock initialized: now 1786450157039048 us; error 39 us; skew 500 ppm
I20260811 12:09:17.039419 8526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:17 dist-test-slave-nj21 krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450157, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:17.042439 8526 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:17.043844 8526 webserver.cc:460] Webserver started at http://127.4.48.254:42199/ using document root <none> and password file <none>
I20260811 12:09:17.044471 8526 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:17.044572 8526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:17.044827 8526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:17.047624 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "7221fd4b869f46519e350a9b885cbc98"
format_stamp: "Formatted at 2026-08-11 12:09:17 on dist-test-slave-nj21"
server_key: "0ad0ed92a260db4b3388b6d35c895209"
server_key_iv: "dbc39b91d13be57f60bf36d5dda6bf62"
server_key_version: "encryptionkey@0"
I20260811 12:09:17.048398 8526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "7221fd4b869f46519e350a9b885cbc98"
format_stamp: "Formatted at 2026-08-11 12:09:17 on dist-test-slave-nj21"
server_key: "0ad0ed92a260db4b3388b6d35c895209"
server_key_iv: "dbc39b91d13be57f60bf36d5dda6bf62"
server_key_version: "encryptionkey@0"
I20260811 12:09:17.053221 8526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:09:17.056443 8556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:17.057641 8526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:17.057782 8526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "7221fd4b869f46519e350a9b885cbc98"
format_stamp: "Formatted at 2026-08-11 12:09:17 on dist-test-slave-nj21"
server_key: "0ad0ed92a260db4b3388b6d35c895209"
server_key_iv: "dbc39b91d13be57f60bf36d5dda6bf62"
server_key_version: "encryptionkey@0"
I20260811 12:09:17.057947 8526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:17.077057 8526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:17.081030 8526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:17.081254 8526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:17.101650 8526 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:43221
I20260811 12:09:17.101698 8648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:43221 every 8 connection(s)
I20260811 12:09:17.102953 8526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:17.106662 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8526
I20260811 12:09:17.106732 8649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:17.106855 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:17.107514 4291 external_mini_cluster.cc:1428] Setting key 20fac7b8884af16119a29cf976a37823
I20260811 12:09:17.120148 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98: Bootstrap starting.
Aug 11 12:09:17 dist-test-slave-nj21 krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:17.123911 8649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:17.125232 8649 log.cc:824] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:17.128064 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98: No bootstrap required, opened a new log
I20260811 12:09:17.128459 8652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:17.109508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54898 (local address 127.4.48.254:43221)
0811 12:09:17.110280 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:17.110292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:17.110332 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:09:17.111891 (+ 1559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:17.111916 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:17.112280 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:17.112614 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:17.114067 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:17.115341 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:17.116652 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:17.117341 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:17.117517 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:17.121492 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:17.121500 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:17.121513 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:17.121553 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:17.124565 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:17.125178 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:17.125185 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:17.125190 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:17.125415 (+ 225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:17.125773 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:17.125778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:17.125781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:17.126332 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:17.126605 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:17.127042 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 12:09:17.127439 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":351,"threads_started":1}
I20260811 12:09:17.136433 8649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } }
I20260811 12:09:17.136771 8649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:17.136871 8649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7221fd4b869f46519e350a9b885cbc98, State: Initialized, Role: FOLLOWER
I20260811 12:09:17.137779 8649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } }
I20260811 12:09:17.137996 8649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:17.138150 8649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:17.138317 8649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:17.140475 8649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } }
I20260811 12:09:17.141042 8649 leader_election.cc:302] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7221fd4b869f46519e350a9b885cbc98; no voters:
I20260811 12:09:17.141680 8649 leader_election.cc:288] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:17.141786 8657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:17.142504 8657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [term 1 LEADER]: Becoming Leader. State: Replica: 7221fd4b869f46519e350a9b885cbc98, State: Running, Role: LEADER
I20260811 12:09:17.143078 8657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } }
I20260811 12:09:17.143770 8649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:17.145928 8658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7221fd4b869f46519e350a9b885cbc98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } } }
I20260811 12:09:17.146325 8658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:17.145998 8662 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7221fd4b869f46519e350a9b885cbc98. Latest consensus state: current_term: 1 leader_uuid: "7221fd4b869f46519e350a9b885cbc98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7221fd4b869f46519e350a9b885cbc98" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43221 } } }
I20260811 12:09:17.146740 8662 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:17.153831 8674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:17.159150 8674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:17.167491 8674 catalog_manager.cc:1347] Generated new cluster ID: 08de3c57932844018184a4176bc9e0c4
I20260811 12:09:17.167596 8674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:17.182739 8674 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:17.184578 8674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:17.193532 8674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7221fd4b869f46519e350a9b885cbc98: Generated new TSK 0
I20260811 12:09:17.194567 8674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
W20260811 12:09:18.065789 4291 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.48.193: real 0.856s user 0.003s sys 0.012s
I20260811 12:09:18.066888 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:18.397907 8692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:18.398192 8692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:18.398319 8692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:18.405386 8692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:18.405514 8692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:18.405669 8692 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:18.415196 8692 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:18.416507 8692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:18.417832 8692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:18.430531 8704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:18.430604 8700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:18.430567 8702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:18.431483 8692 server_base.cc:1034] running on GCE node
I20260811 12:09:18.432132 8692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:18.433036 8692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:18.434231 8692 hybrid_clock.cc:648] HybridClock initialized: now 1786450158434189 us; error 60 us; skew 500 ppm
I20260811 12:09:18.434512 8692 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:18 dist-test-slave-nj21 krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450158, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:18.437590 8692 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:18.439018 8692 webserver.cc:460] Webserver started at http://127.4.48.193:46017/ using document root <none> and password file <none>
I20260811 12:09:18.439648 8692 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:18.439805 8692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:18.440074 8692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:18.442817 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "40b2387443a5403eb302b088defac867"
format_stamp: "Formatted at 2026-08-11 12:09:18 on dist-test-slave-nj21"
server_key: "9a2c0bbcb58d571f81a12266dce08878"
server_key_iv: "9a55e78c6a4c9d8aedefe2ef5a611c33"
server_key_version: "encryptionkey@0"
I20260811 12:09:18.443558 8692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "40b2387443a5403eb302b088defac867"
format_stamp: "Formatted at 2026-08-11 12:09:18 on dist-test-slave-nj21"
server_key: "9a2c0bbcb58d571f81a12266dce08878"
server_key_iv: "9a55e78c6a4c9d8aedefe2ef5a611c33"
server_key_version: "encryptionkey@0"
I20260811 12:09:18.448470 8692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:09:18.451855 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:18.453188 8692 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:09:18.453328 8692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "40b2387443a5403eb302b088defac867"
format_stamp: "Formatted at 2026-08-11 12:09:18 on dist-test-slave-nj21"
server_key: "9a2c0bbcb58d571f81a12266dce08878"
server_key_iv: "9a55e78c6a4c9d8aedefe2ef5a611c33"
server_key_version: "encryptionkey@0"
I20260811 12:09:18.453524 8692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:18.483186 8692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:18.486992 8692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:18.487248 8692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:18.488242 8692 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:18.489563 8692 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:18.489635 8692 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:18.489748 8692 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:18.489807 8692 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:18.523586 8692 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:33393
I20260811 12:09:18.523656 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:33393 every 8 connection(s)
I20260811 12:09:18.524888 8692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:18.527258 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8692
I20260811 12:09:18.527459 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:18.528337 4291 external_mini_cluster.cc:1428] Setting key b00621969fa77d35ab8b084cf6caa252
Aug 11 12:09:18 dist-test-slave-nj21 krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450158, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:18.546271 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:18.527529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:40317 (local address 127.4.48.254:43221)
0811 12:09:18.528082 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:18.528089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:18.529069 (+ 980us) server_negotiation.cc:407] Connection header received
0811 12:09:18.530909 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:18.530916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:18.531003 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:18.531133 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:18.532594 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:18.533611 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:18.535137 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:18.536501 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:18.536642 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:18.540348 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:18.540354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:18.540357 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:18.540393 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:18.543079 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:18.543924 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:18.543928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:18.543931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:18.544066 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:18.544564 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:18.544569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:18.544571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:18.544919 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:18.545040 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:18.545807 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 12:09:18.545978 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":267,"threads_started":1}
I20260811 12:09:18.547711 8909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:18.528111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43221 (local address 127.4.48.193:40317)
0811 12:09:18.528888 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:18.528942 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:18.530644 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:18.531300 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:18.531308 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:18.531847 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:18.532404 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:18.532421 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:18.533778 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:18.533782 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:18.534990 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:18.535003 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:18.536702 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:18.536706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:18.536913 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:18.537562 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:18.537583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:18.540075 (+ 2492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:18.543228 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:18.543237 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:18.543251 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:18.543808 (+ 557us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:18.544191 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:18.544195 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:18.544199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:18.544441 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:18.545036 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:18.545043 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:18.545453 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 12:09:18.545730 (+ 277us) client_negotiation.cc:240] Negotiation successful
0811 12:09:18.546296 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":332,"threads_started":1}
I20260811 12:09:18.549161 8904 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43221
I20260811 12:09:18.549470 8904 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
I20260811 12:09:18.550220 8904 heartbeater.cc:511] Master 127.4.48.254:43221 requested a full tablet report, sending...
I20260811 12:09:18.552285 8586 ts_manager.cc:194] Registered new tserver with Master: 40b2387443a5403eb302b088defac867 (127.4.48.193:33393)
I20260811 12:09:18.553730 8586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:40317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
W20260811 12:09:19.002379 4291 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.4.48.194: real 0.471s user 0.000s sys 0.015s
I20260811 12:09:19.003131 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:19.330178 8912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:19.330466 8912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:19.330596 8912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:19.337661 8912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:19.337779 8912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:19.337929 8912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:19.347484 8912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:19.348748 8912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:19.350055 8912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:19.363121 8923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:19.363202 8926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:19.363161 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:19.364063 8912 server_base.cc:1034] running on GCE node
I20260811 12:09:19.364645 8912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:19.365559 8912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:19.366744 8912 hybrid_clock.cc:648] HybridClock initialized: now 1786450159366702 us; error 66 us; skew 500 ppm
I20260811 12:09:19.367033 8912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:19 dist-test-slave-nj21 krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:19.370059 8912 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:19.371485 8912 webserver.cc:460] Webserver started at http://127.4.48.194:43517/ using document root <none> and password file <none>
I20260811 12:09:19.372174 8912 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:19.372290 8912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:19.372596 8912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:19.375411 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "5e5f3c607d5c4d8c919c919b99ed45c3"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "320a0446ffe99b71ec72518d41e2adf6"
server_key_iv: "2383e26ac10f8d313fc0a706ffa312d2"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.376312 8912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "5e5f3c607d5c4d8c919c919b99ed45c3"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "320a0446ffe99b71ec72518d41e2adf6"
server_key_iv: "2383e26ac10f8d313fc0a706ffa312d2"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.381247 8912 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:09:19.384513 8943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:19.385810 8912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:19.385965 8912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "5e5f3c607d5c4d8c919c919b99ed45c3"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "320a0446ffe99b71ec72518d41e2adf6"
server_key_iv: "2383e26ac10f8d313fc0a706ffa312d2"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.386137 8912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:19.407452 8912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:19.411339 8912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:19.411607 8912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:19.412590 8912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:19.413894 8912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:19.413975 8912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:19.414096 8912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:19.414151 8912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:19.444751 8912 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:44471
I20260811 12:09:19.444885 9142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:44471 every 8 connection(s)
I20260811 12:09:19.446081 8912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:19.453661 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8912
I20260811 12:09:19.453845 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:19.454264 4291 external_mini_cluster.cc:1428] Setting key 18202e6cd5c3b15bc6587ba76bc887dc
Aug 11 12:09:19 dist-test-slave-nj21 krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:19.467021 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:19.448585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:45433 (local address 127.4.48.254:43221)
0811 12:09:19.449042 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:19.449046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:19.449882 (+ 836us) server_negotiation.cc:407] Connection header received
0811 12:09:19.451848 (+ 1966us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:19.451857 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:19.451961 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:19.452079 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:19.453919 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.454776 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.456690 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.457158 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.457251 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:19.461109 (+ 3858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:19.461116 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:19.461121 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:19.461159 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:19.463974 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:19.464732 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:19.464736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:19.464738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:19.464879 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:19.465307 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:19.465311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:19.465314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:19.465567 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:19.465678 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:19.466423 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 12:09:19.466652 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":243,"threads_started":1}
I20260811 12:09:19.468037 9148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:19.449002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43221 (local address 127.4.48.194:45433)
0811 12:09:19.449726 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:19.449764 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:19.451415 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:19.452445 (+ 1030us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:19.452456 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:19.453126 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:19.453676 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:19.453697 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.454951 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.454956 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:19.456553 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:19.456567 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.457388 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.457395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:19.457689 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:19.458406 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:19.458438 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:19.460893 (+ 2455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:19.464101 (+ 3208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:19.464110 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:19.464127 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:19.464608 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:19.464978 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:19.464983 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:19.464986 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:19.465220 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:19.465686 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:19.465694 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:19.466065 (+ 371us) client_negotiation.cc:769] Sending connection context
0811 12:09:19.466303 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 12:09:19.466804 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":275,"threads_started":1}
I20260811 12:09:19.469316 9143 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43221
I20260811 12:09:19.469614 9143 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:19.470290 9143 heartbeater.cc:511] Master 127.4.48.254:43221 requested a full tablet report, sending...
I20260811 12:09:19.471735 8586 ts_manager.cc:194] Registered new tserver with Master: 5e5f3c607d5c4d8c919c919b99ed45c3 (127.4.48.194:44471)
I20260811 12:09:19.472520 8586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:45433
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:19.500885 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:09:19.556535 8904 heartbeater.cc:503] Master 127.4.48.254:43221 was elected leader, sending a full tablet report...
W20260811 12:09:19.824083 9153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:19.824333 9153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:19.824424 9153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:19.831347 9153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:19.831435 9153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:19.831549 9153 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:19.841131 9153 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43221
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:19.842252 9153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:19.843433 9153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:19.857753 9162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:19.857853 9163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:19.857829 9165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:19.858763 9153 server_base.cc:1034] running on GCE node
I20260811 12:09:19.859321 9153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:19.860235 9153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:19.861424 9153 hybrid_clock.cc:648] HybridClock initialized: now 1786450159861378 us; error 33 us; skew 500 ppm
I20260811 12:09:19.861711 9153 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:19 dist-test-slave-nj21 krb5kdc[8509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:19.864679 9153 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:19.865960 9153 webserver.cc:460] Webserver started at http://127.4.48.195:35059/ using document root <none> and password file <none>
I20260811 12:09:19.866613 9153 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:19.866711 9153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:19.866978 9153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:19.869776 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "30a6b8e19b7340c196d2f6b00751fe7c"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "ee6141067e2a42a239e46622e507ad0f"
server_key_iv: "440b4de654d928328ba54fcaccd0f4c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.870493 9153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "30a6b8e19b7340c196d2f6b00751fe7c"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "ee6141067e2a42a239e46622e507ad0f"
server_key_iv: "440b4de654d928328ba54fcaccd0f4c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.875290 9153 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 12:09:19.878576 9177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:19.879839 9153 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:09:19.879992 9153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "30a6b8e19b7340c196d2f6b00751fe7c"
format_stamp: "Formatted at 2026-08-11 12:09:19 on dist-test-slave-nj21"
server_key: "ee6141067e2a42a239e46622e507ad0f"
server_key_iv: "440b4de654d928328ba54fcaccd0f4c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:19.880164 9153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:19.897626 9153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:19.901130 9153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:19.901373 9153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:19.902226 9153 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:19.903461 9153 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:19.903537 9153 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:19.903656 9153 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:19.903744 9153 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 12:09:19.933158 9153 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:37965
I20260811 12:09:19.933238 9367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:37965 every 8 connection(s)
I20260811 12:09:19.934347 9153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:19.938661 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9153
I20260811 12:09:19.938828 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:19.939260 4291 external_mini_cluster.cc:1428] Setting key c44b6b2c5400688813ce4c08cf2d8725
Aug 11 12:09:19 dist-test-slave-nj21 krb5kdc[8509](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450159, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:19.953471 9147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:19.936888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:44617 (local address 127.4.48.254:43221)
0811 12:09:19.937076 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:19.937080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:19.938147 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 12:09:19.940206 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:19.940212 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:19.940328 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:19.940426 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:19.942028 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.942788 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.944405 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.944809 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.944897 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:19.948192 (+ 3295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:19.948196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:19.948200 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:19.948238 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:19.950625 (+ 2387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:19.951306 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:19.951310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:19.951312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:19.951416 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:19.951791 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:19.951794 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:19.951796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:19.952052 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:19.952169 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:19.952932 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 12:09:19.953157 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:09:19.954522 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:19.937267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43221 (local address 127.4.48.195:44617)
0811 12:09:19.937990 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:19.938034 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:19.939897 (+ 1863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:19.940677 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:19.940685 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:19.941292 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:19.941854 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:19.941879 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.942935 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.942939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:19.944291 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:19.944304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:19.945000 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:19.945008 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:19.945237 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:19.945822 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:19.945845 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:19.948002 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:19.950769 (+ 2767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:19.950776 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:19.950787 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:19.951212 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:19.951491 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:19.951494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:19.951496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:19.951702 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:19.952151 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:19.952156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:19.952484 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 12:09:19.952782 (+ 298us) client_negotiation.cc:240] Negotiation successful
0811 12:09:19.953255 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":271,"threads_started":1}
I20260811 12:09:19.955713 9368 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43221
I20260811 12:09:19.955979 9368 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:19.956674 9368 heartbeater.cc:511] Master 127.4.48.254:43221 requested a full tablet report, sending...
I20260811 12:09:19.957991 8586 ts_manager.cc:194] Registered new tserver with Master: 30a6b8e19b7340c196d2f6b00751fe7c (127.4.48.195:37965)
I20260811 12:09:19.958894 8586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:44617
I20260811 12:09:19.966310 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:09:20.001874 9147 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:19.999389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54906 (local address 127.4.48.254:43221)
0811 12:09:19.999574 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:19.999581 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:20.000021 (+ 440us) server_negotiation.cc:407] Connection header received
0811 12:09:20.000217 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:20.000221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:20.000289 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:20.000384 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:20.001612 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54906: BlockingRecv error: recv got EOF from 127.0.0.1:54906 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:09:20.005142 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8692
W20260811 12:09:20.007879 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8692: Not authorized: Client connection negotiation failed: client connection to 127.4.48.193:33393: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:09:20.008030 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8692
I20260811 12:09:20.029131 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8912
W20260811 12:09:20.031559 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8912: Not authorized: Client connection negotiation failed: client connection to 127.4.48.194:44471: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:09:20.031759 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8912
I20260811 12:09:20.050132 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9153
W20260811 12:09:20.052781 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9153: Not authorized: Client connection negotiation failed: client connection to 127.4.48.195:37965: server requires authentication, but client does not have Kerberos credentials available
I20260811 12:09:20.052911 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9153
I20260811 12:09:20.072124 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8526
I20260811 12:09:20.073295 8640 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:20.174225 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8526
2026-08-11T12:09:20Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4891 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:20 dist-test-slave-nj21 krb5kdc[9395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:20 dist-test-slave-nj21 krb5kdc[9395](info): set up 2 sockets
Aug 11 12:09:20 dist-test-slave-nj21 krb5kdc[9395](info): commencing operation
krb5kdc: starting...
W20260811 12:09:21.394388 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:21 dist-test-slave-nj21 krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:21.592720 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39111
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:46367
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:21.914318 9414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:21.914583 9414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:21.914724 9414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:21.921734 9414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:21.921829 9414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:21.921900 9414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:21.921969 9414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:21.922032 9414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:21.931449 9414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:46367
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:21.932744 9414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:21.933991 9414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:21.945130 9420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:21.945236 9426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:21.945307 9423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:21.946306 9414 server_base.cc:1034] running on GCE node
I20260811 12:09:21.947021 9414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:21.948354 9414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:21.949561 9414 hybrid_clock.cc:648] HybridClock initialized: now 1786450161949523 us; error 30 us; skew 500 ppm
I20260811 12:09:21.949903 9414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:21 dist-test-slave-nj21 krb5kdc[9395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450161, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:21.952975 9414 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:21.954326 9414 webserver.cc:460] Webserver started at http://127.4.48.254:43487/ using document root <none> and password file <none>
I20260811 12:09:21.954977 9414 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:21.955101 9414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:21.955411 9414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:21.958297 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "489dd8df0dd44b2a88def5e7d005d659"
format_stamp: "Formatted at 2026-08-11 12:09:21 on dist-test-slave-nj21"
server_key: "0e76c0912d505472cafb7c1a8e101082"
server_key_iv: "67744290227e2a543e27b26c7adccb1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:21.959074 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "489dd8df0dd44b2a88def5e7d005d659"
format_stamp: "Formatted at 2026-08-11 12:09:21 on dist-test-slave-nj21"
server_key: "0e76c0912d505472cafb7c1a8e101082"
server_key_iv: "67744290227e2a543e27b26c7adccb1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:21.964109 9414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 12:09:21.967262 9440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:21.968494 9414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:21.968652 9414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "489dd8df0dd44b2a88def5e7d005d659"
format_stamp: "Formatted at 2026-08-11 12:09:21 on dist-test-slave-nj21"
server_key: "0e76c0912d505472cafb7c1a8e101082"
server_key_iv: "67744290227e2a543e27b26c7adccb1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:21.968822 9414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:21.994791 9414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:21.998747 9414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:21.999004 9414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:22.016014 9414 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39111
I20260811 12:09:22.016058 9532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39111 every 8 connection(s)
I20260811 12:09:22.017266 9414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:22.019271 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9414
I20260811 12:09:22.019444 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:22.020139 4291 external_mini_cluster.cc:1428] Setting key 245ceabb077a7e58e0d15630a43a3aa8
I20260811 12:09:22.021001 9535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:09:22 dist-test-slave-nj21 krb5kdc[9395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:22.034665 9535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659: Bootstrap starting.
I20260811 12:09:22.037696 9535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:22.038887 9535 log.cc:824] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:22.042099 9535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659: No bootstrap required, opened a new log
I20260811 12:09:22.042286 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:22.021823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53198 (local address 127.4.48.254:39111)
0811 12:09:22.022581 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:22.022592 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:22.022630 (+ 38us) server_negotiation.cc:407] Connection header received
0811 12:09:22.024165 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:22.024185 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:22.024545 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:22.024855 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:22.026739 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:22.028112 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:22.029813 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:22.030454 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:22.030579 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:22.034930 (+ 4351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:22.034938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:22.034950 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:22.034983 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:22.037761 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:22.038506 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:22.038512 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:22.038518 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:22.038684 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:22.039171 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:22.039176 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:22.039180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:22.039878 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:22.040110 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:22.040608 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 12:09:22.041060 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":329,"threads_started":1}
I20260811 12:09:22.045811 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:09:22.051322 9535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } }
I20260811 12:09:22.051765 9535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:22.051872 9535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 489dd8df0dd44b2a88def5e7d005d659, State: Initialized, Role: FOLLOWER
I20260811 12:09:22.052508 9535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } }
I20260811 12:09:22.052784 9535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:22.052917 9535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:22.053086 9535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:22.055773 9535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } }
I20260811 12:09:22.056313 9535 leader_election.cc:302] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 489dd8df0dd44b2a88def5e7d005d659; no voters:
I20260811 12:09:22.056917 9535 leader_election.cc:288] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:22.057021 9545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:22.058120 9545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [term 1 LEADER]: Becoming Leader. State: Replica: 489dd8df0dd44b2a88def5e7d005d659, State: Running, Role: LEADER
I20260811 12:09:22.058662 9545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } }
I20260811 12:09:22.059170 9535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:22.061375 9550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 489dd8df0dd44b2a88def5e7d005d659. Latest consensus state: current_term: 1 leader_uuid: "489dd8df0dd44b2a88def5e7d005d659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } } }
I20260811 12:09:22.061704 9550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:22.062219 9548 sys_catalog.cc:455] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "489dd8df0dd44b2a88def5e7d005d659" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "489dd8df0dd44b2a88def5e7d005d659" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39111 } } }
I20260811 12:09:22.062507 9548 sys_catalog.cc:458] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:22.064270 9559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:22.069967 9559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:22.079932 9576 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:09:22.082342 9559 catalog_manager.cc:1347] Generated new cluster ID: 5ef28ddaea0843e8a2504ee915af21c4
I20260811 12:09:22.082520 9559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:22.100822 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:22.089778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53214 (local address 127.4.48.254:39111)
0811 12:09:22.089948 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:22.089954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:22.092308 (+ 2354us) server_negotiation.cc:407] Connection header received
0811 12:09:22.092496 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:22.092502 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:22.092581 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:22.092687 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:22.093939 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:22.095093 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:22.097059 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:22.097711 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:22.097839 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:22.099024 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:22.099030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:22.099035 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:22.099961 (+ 926us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:22.100019 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:22.100062 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:22.100164 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:22.100322 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 12:09:22.100412 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":49}
I20260811 12:09:22.107945 9559 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:22.109777 9559 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:22.126343 9559 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 489dd8df0dd44b2a88def5e7d005d659: Generated new TSK 0
I20260811 12:09:22.127306 9559 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:09:22.143896 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9414
I20260811 12:09:22.145215 9524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:22.234694 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9414
2026-08-11T12:09:22Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2055 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:22 dist-test-slave-nj21 krb5kdc[9595](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:22 dist-test-slave-nj21 krb5kdc[9595](info): set up 2 sockets
Aug 11 12:09:22 dist-test-slave-nj21 krb5kdc[9595](info): commencing operation
krb5kdc: starting...
W20260811 12:09:23.445876 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:23 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:23.641141 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44723
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:44723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:23.970335 9615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:23.970623 9615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:23.970770 9615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:23.977736 9615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:23.977843 9615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:23.977928 9615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:23.978003 9615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:23.978065 9615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:23.987452 9615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:44723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:23.988682 9615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:23.989965 9615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:24.001303 9633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:24.001338 9637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:24.001338 9640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:24.002198 9615 server_base.cc:1034] running on GCE node
I20260811 12:09:24.002924 9615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:24.004226 9615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:24.005411 9615 hybrid_clock.cc:648] HybridClock initialized: now 1786450164005358 us; error 41 us; skew 500 ppm
I20260811 12:09:24.005721 9615 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:24 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450164, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:24.008682 9615 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:24.010110 9615 webserver.cc:460] Webserver started at http://127.4.48.254:43695/ using document root <none> and password file <none>
I20260811 12:09:24.010736 9615 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:24.010848 9615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:24.011126 9615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:24.013960 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "40db388179b34766acfc43f1800b5fab"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "4bc799a4048fa70a1856839f57201e80"
server_key_iv: "3318ff75289ba4f9e09a22eaf665bb57"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.014676 9615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "40db388179b34766acfc43f1800b5fab"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "4bc799a4048fa70a1856839f57201e80"
server_key_iv: "3318ff75289ba4f9e09a22eaf665bb57"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.019606 9615 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:09:24.022771 9661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:24.023990 9615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 12:09:24.024142 9615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "40db388179b34766acfc43f1800b5fab"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "4bc799a4048fa70a1856839f57201e80"
server_key_iv: "3318ff75289ba4f9e09a22eaf665bb57"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.024319 9615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:24.035619 9615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:24.039156 9615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:24.039407 9615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:24.056217 9615 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:44723
I20260811 12:09:24.056308 9746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:44723 every 8 connection(s)
I20260811 12:09:24.057478 9615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:24.061173 9747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:24.067814 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9615
I20260811 12:09:24.068027 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:24.068651 4291 external_mini_cluster.cc:1428] Setting key 61edb38e2ea58d20327ca9b57d0a34aa
I20260811 12:09:24.072075 9747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab: Bootstrap starting.
I20260811 12:09:24.075862 9747 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:24.077083 9747 log.cc:824] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:24.080242 9747 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab: No bootstrap required, opened a new log
Aug 11 12:09:24 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:24.090898 9747 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } }
I20260811 12:09:24.091367 9747 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:24.091497 9747 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40db388179b34766acfc43f1800b5fab, State: Initialized, Role: FOLLOWER
I20260811 12:09:24.092221 9747 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } }
I20260811 12:09:24.092531 9747 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:24.092746 9750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:24.070501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45396 (local address 127.4.48.254:44723)
0811 12:09:24.071227 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:24.071238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:24.071279 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:09:24.072971 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:24.072994 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:24.073336 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:24.073643 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:24.075570 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.076962 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.078585 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.079255 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.079384 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:24.084080 (+ 4696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:24.084096 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:24.084116 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:24.084169 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:24.088224 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:24.088853 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:24.088863 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:24.088872 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:24.089108 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:24.089484 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:24.089494 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:24.089498 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:24.090218 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:24.090532 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:24.090850 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:09:24.091303 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":350,"threads_started":1}
I20260811 12:09:24.092724 9747 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:24.092978 9747 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:24.095417 9747 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } }
I20260811 12:09:24.095981 9747 leader_election.cc:302] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40db388179b34766acfc43f1800b5fab; no voters:
I20260811 12:09:24.096557 9747 leader_election.cc:288] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:24.096650 9753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:24.098021 9753 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [term 1 LEADER]: Becoming Leader. State: Replica: 40db388179b34766acfc43f1800b5fab, State: Running, Role: LEADER
I20260811 12:09:24.098522 9753 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } }
I20260811 12:09:24.098623 9747 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:24.101018 9756 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40db388179b34766acfc43f1800b5fab. Latest consensus state: current_term: 1 leader_uuid: "40db388179b34766acfc43f1800b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } } }
I20260811 12:09:24.101362 9756 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:24.101886 9755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40db388179b34766acfc43f1800b5fab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40db388179b34766acfc43f1800b5fab" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44723 } } }
I20260811 12:09:24.102201 9755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:24.103626 9766 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:24.108829 9766 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:24.118686 9766 catalog_manager.cc:1347] Generated new cluster ID: 2710f0f043504e90827ebd141d768504
I20260811 12:09:24.118842 9766 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:24.178490 9766 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:24.180454 9766 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:24.192214 9766 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40db388179b34766acfc43f1800b5fab: Generated new TSK 0
I20260811 12:09:24.193225 9766 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:24.260550 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:24.584322 9792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:24.584576 9792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:24.584666 9792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:24.592090 9792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:24.592192 9792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:24.592301 9792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:24.601797 9792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:24.603055 9792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:24.604359 9792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:24.616986 9800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:24.617944 9810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:24.617996 9802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:24.618834 9792 server_base.cc:1034] running on GCE node
I20260811 12:09:24.619445 9792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:24.620425 9792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:24.621628 9792 hybrid_clock.cc:648] HybridClock initialized: now 1786450164621547 us; error 69 us; skew 500 ppm
I20260811 12:09:24.621910 9792 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:24 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450164, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:24.625042 9792 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:24.626385 9792 webserver.cc:460] Webserver started at http://127.4.48.193:43025/ using document root <none> and password file <none>
I20260811 12:09:24.627053 9792 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:24.627199 9792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:24.627523 9792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:24.630571 9792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "ed346e82b36e4787a82ed4c9296318ba"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "d672bd663da8e345c71cf0848f043879"
server_key_iv: "bb89eb6ce2b7961e448bb13b4bf7c03f"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.631342 9792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "ed346e82b36e4787a82ed4c9296318ba"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "d672bd663da8e345c71cf0848f043879"
server_key_iv: "bb89eb6ce2b7961e448bb13b4bf7c03f"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.636309 9792 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 12:09:24.639660 9818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:24.640929 9792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:24.641085 9792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ed346e82b36e4787a82ed4c9296318ba"
format_stamp: "Formatted at 2026-08-11 12:09:24 on dist-test-slave-nj21"
server_key: "d672bd663da8e345c71cf0848f043879"
server_key_iv: "bb89eb6ce2b7961e448bb13b4bf7c03f"
server_key_version: "encryptionkey@0"
I20260811 12:09:24.641258 9792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:24.665922 9792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:24.669529 9792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:24.669812 9792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:24.670704 9792 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:24.672046 9792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:24.672120 9792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:24.672215 9792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:24.672273 9792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:24.702255 9792 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:46723
I20260811 12:09:24.702313 10002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:46723 every 8 connection(s)
I20260811 12:09:24.703441 9792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:24.709959 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9792
I20260811 12:09:24.710130 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:24.710745 4291 external_mini_cluster.cc:1428] Setting key fc58974c1782c96fed36daaea52e1253
Aug 11 12:09:24 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450164, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:24.722394 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:24.705727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:46035 (local address 127.4.48.254:44723)
0811 12:09:24.706129 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:24.706134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:24.706969 (+ 835us) server_negotiation.cc:407] Connection header received
0811 12:09:24.708804 (+ 1835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:24.708810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:24.708890 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:24.708987 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:24.710483 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.711246 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.713060 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.713454 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.713565 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:24.716789 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:24.716795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:24.716800 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:24.716837 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:24.719598 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:24.720286 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:24.720290 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:24.720293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:24.720409 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:24.720853 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:24.720856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:24.720858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:24.721096 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:24.721198 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:24.721985 (+ 787us) server_negotiation.cc:299] Negotiation successful
0811 12:09:24.722142 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":209,"threads_started":1}
I20260811 12:09:24.723613 10008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:24.706133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44723 (local address 127.4.48.193:46035)
0811 12:09:24.706815 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:24.706857 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:24.708541 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:24.709221 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:24.709229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:24.709718 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:24.710318 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:24.710337 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.711405 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.711409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:24.712907 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:24.712920 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:24.713669 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:24.713676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:24.713889 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:24.714482 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:24.714505 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:24.716574 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:24.719753 (+ 3179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:24.719760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:24.719771 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:24.720158 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:24.720501 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:24.720505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:24.720508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:24.720733 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:24.721214 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:24.721221 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:24.721605 (+ 384us) client_negotiation.cc:769] Sending connection context
0811 12:09:24.721908 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:09:24.722381 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":266,"threads_started":1}
I20260811 12:09:24.725080 10003 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44723
I20260811 12:09:24.725379 10003 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:24.726243 10003 heartbeater.cc:511] Master 127.4.48.254:44723 requested a full tablet report, sending...
I20260811 12:09:24.728607 9686 ts_manager.cc:194] Registered new tserver with Master: ed346e82b36e4787a82ed4c9296318ba (127.4.48.193:46723)
I20260811 12:09:24.730633 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:46035
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:24.760965 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:25.078119 10013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:25.078390 10013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:25.078532 10013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:25.085469 10013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:25.085582 10013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:25.085721 10013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:25.095201 10013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:25.096464 10013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:25.097771 10013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:25.110229 10022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:25.110253 10030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:25.110292 10026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:25.111258 10013 server_base.cc:1034] running on GCE node
I20260811 12:09:25.111856 10013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:25.112709 10013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:25.113926 10013 hybrid_clock.cc:648] HybridClock initialized: now 1786450165113883 us; error 33 us; skew 500 ppm
I20260811 12:09:25.114229 10013 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:25.117298 10013 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:25.118649 10013 webserver.cc:460] Webserver started at http://127.4.48.194:33947/ using document root <none> and password file <none>
I20260811 12:09:25.119266 10013 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:25.119369 10013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:25.119619 10013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:25.122417 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "74597701d75a4c6291c102fddb8f9500"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "ad91291a1f29c6a095da1af4d332cdba"
server_key_iv: "f95c0dafd576612cd1522d3ae5bd70b3"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.123121 10013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "74597701d75a4c6291c102fddb8f9500"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "ad91291a1f29c6a095da1af4d332cdba"
server_key_iv: "f95c0dafd576612cd1522d3ae5bd70b3"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.128039 10013 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:09:25.131376 10049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.132637 10013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:25.132774 10013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "74597701d75a4c6291c102fddb8f9500"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "ad91291a1f29c6a095da1af4d332cdba"
server_key_iv: "f95c0dafd576612cd1522d3ae5bd70b3"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.132931 10013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:25.158569 10013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:25.162565 10013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:25.162839 10013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:25.163793 10013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:25.165073 10013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:25.165144 10013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.165266 10013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:25.165316 10013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.195293 10013 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:45563
I20260811 12:09:25.195341 10244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:45563 every 8 connection(s)
I20260811 12:09:25.196528 10013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:25.198701 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10013
I20260811 12:09:25.198880 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:25.199446 4291 external_mini_cluster.cc:1428] Setting key 87bb03303503ec8abff030def918e790
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:25.216061 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.199212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:34561 (local address 127.4.48.254:44723)
0811 12:09:25.199425 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.199432 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.200659 (+ 1227us) server_negotiation.cc:407] Connection header received
0811 12:09:25.202537 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.202543 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.202624 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.202731 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.204106 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.205033 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.206475 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.207002 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.207119 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.210607 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.210611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.210614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.210647 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.213198 (+ 2551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.213890 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.213896 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.213900 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.214028 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.214431 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.214436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.214439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.214688 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.214833 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.215475 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.215727 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:09:25.216952 10250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.199712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44723 (local address 127.4.48.194:34561)
0811 12:09:25.200510 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:25.200556 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:25.202254 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:25.202890 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:25.202897 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:25.203363 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:25.203951 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:25.203969 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.205151 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.205154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:25.206332 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:25.206348 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.207181 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.207186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:25.207393 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.208035 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:25.208057 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:25.210390 (+ 2333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:25.213337 (+ 2947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:25.213346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:25.213360 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:25.213770 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:25.214131 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:25.214136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:25.214139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:25.214315 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:25.214816 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:25.214824 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:25.215114 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 12:09:25.215341 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:09:25.215860 (+ 519us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":327,"threads_started":1}
I20260811 12:09:25.218283 10245 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44723
I20260811 12:09:25.218628 10245 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:25.219525 10245 heartbeater.cc:511] Master 127.4.48.254:44723 requested a full tablet report, sending...
I20260811 12:09:25.221060 9686 ts_manager.cc:194] Registered new tserver with Master: 74597701d75a4c6291c102fddb8f9500 (127.4.48.194:45563)
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
I20260811 12:09:25.221873 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:34561
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:25.246992 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:25.560329 10254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:25.560608 10254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:25.560722 10254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:25.567847 10254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:25.567952 10254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:25.568112 10254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:25.577208 10254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44723
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:25.578411 10254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:25.579686 10254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:25.592427 10264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:25.592638 10262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:25.592842 10261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:25.593750 10254 server_base.cc:1034] running on GCE node
I20260811 12:09:25.594275 10254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:25.595084 10254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:25.596275 10254 hybrid_clock.cc:648] HybridClock initialized: now 1786450165596229 us; error 31 us; skew 500 ppm
I20260811 12:09:25.596572 10254 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:25.603550 10254 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:25.604880 10254 webserver.cc:460] Webserver started at http://127.4.48.195:35347/ using document root <none> and password file <none>
I20260811 12:09:25.605508 10254 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:25.605625 10254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:25.605917 10254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:25.608830 10254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "b1a3ce684e7945d1aa7dd17ad4f6cd13"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "7a30b5837d54188b4f744c14c2c4a50a"
server_key_iv: "bcb9cca216a0c2f3b723d89aaa929286"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.609596 10254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "b1a3ce684e7945d1aa7dd17ad4f6cd13"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "7a30b5837d54188b4f744c14c2c4a50a"
server_key_iv: "bcb9cca216a0c2f3b723d89aaa929286"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.614701 10254 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:09:25.617945 10277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.619115 10254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:09:25.619275 10254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "b1a3ce684e7945d1aa7dd17ad4f6cd13"
format_stamp: "Formatted at 2026-08-11 12:09:25 on dist-test-slave-nj21"
server_key: "7a30b5837d54188b4f744c14c2c4a50a"
server_key_iv: "bcb9cca216a0c2f3b723d89aaa929286"
server_key_version: "encryptionkey@0"
I20260811 12:09:25.619454 10254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:25.650768 10254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:25.654803 10254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:25.655077 10254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:25.656065 10254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:25.657709 10254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:25.657851 10254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.658070 10254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:25.658191 10254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:25.693820 10254 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:46613
I20260811 12:09:25.693887 10469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:46613 every 8 connection(s)
I20260811 12:09:25.695127 10254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:25.704943 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10254
I20260811 12:09:25.705111 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:25.705646 4291 external_mini_cluster.cc:1428] Setting key 501a9fa9577e32a1655e663ee8ee8f20
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:25.715072 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.697677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:60551 (local address 127.4.48.254:44723)
0811 12:09:25.697862 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.697866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.698966 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 12:09:25.700812 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.700817 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.700890 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.700976 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.702673 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.703435 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.705732 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.706380 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.706500 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.710257 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.710262 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.710265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.710299 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.712575 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.713185 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.713188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.713191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.713308 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.713668 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.713671 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.713673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.713939 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.714044 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.714641 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.714827 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:09:25.716017 10473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.698096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44723 (local address 127.4.48.195:60551)
0811 12:09:25.698810 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:25.698850 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:25.700563 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:25.701161 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:25.701169 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:25.701749 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:25.702503 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:25.702531 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.703592 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.703597 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:25.705541 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:25.705556 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.706602 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.706611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:25.706962 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.707709 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:25.707738 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:25.710070 (+ 2332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:25.712701 (+ 2631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:25.712708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:25.712717 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:25.713097 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:25.713403 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:25.713406 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:25.713409 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:25.713587 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:25.714025 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:25.714030 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:25.714311 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 12:09:25.714513 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 12:09:25.714862 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":282,"threads_started":1}
I20260811 12:09:25.717367 10470 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44723
I20260811 12:09:25.717660 10470 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:25.718357 10470 heartbeater.cc:511] Master 127.4.48.254:44723 requested a full tablet report, sending...
I20260811 12:09:25.720006 9686 ts_manager.cc:194] Registered new tserver with Master: b1a3ce684e7945d1aa7dd17ad4f6cd13 (127.4.48.195:46613)
I20260811 12:09:25.721047 9686 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:60551
I20260811 12:09:25.721256 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:25.733659 10003 heartbeater.cc:503] Master 127.4.48.254:44723 was elected leader, sending a full tablet report...
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:25.772950 10006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.759839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.4.48.254:44723)
0811 12:09:25.760035 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.760042 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.760460 (+ 418us) server_negotiation.cc:407] Connection header received
0811 12:09:25.760637 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.760641 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.760707 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.760792 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.762109 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.762890 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.764162 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.764576 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.764687 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.768456 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.768461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.768464 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.768499 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.770738 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.771331 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.771334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.771337 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.771469 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.771833 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.771836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.771838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.772156 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.772336 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.772558 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.772717 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 12:09:25.773509 9686 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41160
I20260811 12:09:25.775156 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9792
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:25.790127 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.776283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56046 (local address 127.4.48.193:46723)
0811 12:09:25.776854 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.776861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.776884 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:09:25.776937 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.776944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.777137 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.777299 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.778358 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.779349 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.780499 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.781052 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.781164 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.784660 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.784671 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.784684 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.784725 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.787378 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.788094 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.788101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.788103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.788233 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.788603 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.788606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.788608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.788955 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.789238 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.789553 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.789814 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":278,"threads_started":1}
W20260811 12:09:25.791322 9991 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:56046
W20260811 12:09:25.792378 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9792: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:25.792569 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9792
I20260811 12:09:25.815991 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10013
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:25.831209 10488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.816791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34172 (local address 127.4.48.194:45563)
0811 12:09:25.817323 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.817329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.817356 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:09:25.817420 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.817427 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.817636 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.817763 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.818968 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.820036 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.821273 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.821808 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.821920 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.825960 (+ 4040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.825973 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.825987 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.826021 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.828738 (+ 2717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.829334 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.829341 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.829343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.829480 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.829764 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.829767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.829769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.830131 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.830373 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.830653 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.830847 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":261,"threads_started":1}
W20260811 12:09:25.832547 10234 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34172
W20260811 12:09:25.833578 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10013: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:25.833703 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10013
I20260811 12:09:25.857477 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10254
Aug 11 12:09:25 dist-test-slave-nj21 krb5kdc[9595](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450165, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:25.872202 10491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:25.858300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60892 (local address 127.4.48.195:46613)
0811 12:09:25.858930 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:25.858937 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:25.858967 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:09:25.859025 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:25.859031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:25.859233 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:25.859358 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:25.860612 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.861672 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.862766 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:25.863319 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:25.863413 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:25.867307 (+ 3894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:25.867318 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:25.867331 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:25.867365 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:25.870006 (+ 2641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.870540 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.870546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.870548 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.870655 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:25.870919 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:25.870922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:25.870924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:25.871240 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:25.871478 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:25.871638 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 12:09:25.871925 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":292,"threads_started":1}
W20260811 12:09:25.873404 10465 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60892
W20260811 12:09:25.874420 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10254: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:09:25.874571 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10254
I20260811 12:09:25.898856 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9615
I20260811 12:09:25.900104 9741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:26.003177 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9615
2026-08-11T12:09:26Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3775 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:26 dist-test-slave-nj21 krb5kdc[10501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:26 dist-test-slave-nj21 krb5kdc[10501](info): set up 2 sockets
Aug 11 12:09:26 dist-test-slave-nj21 krb5kdc[10501](info): commencing operation
krb5kdc: starting...
W20260811 12:09:27.222002 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:27 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:27.427119 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39725
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:27.750977 10525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:27.751289 10525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:27.751428 10525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:27.758453 10525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:27.758558 10525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:27.758639 10525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:27.758720 10525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:27.758781 10525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:27.768205 10525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:27.769484 10525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:27.770737 10525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:27.781692 10530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:27.781739 10536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:27.781769 10533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:27.782729 10525 server_base.cc:1034] running on GCE node
I20260811 12:09:27.783512 10525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:27.784839 10525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:27.786041 10525 hybrid_clock.cc:648] HybridClock initialized: now 1786450167785993 us; error 39 us; skew 500 ppm
I20260811 12:09:27.786370 10525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:27 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450167, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:27.789511 10525 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:27.790951 10525 webserver.cc:460] Webserver started at http://127.4.48.254:41077/ using document root <none> and password file <none>
I20260811 12:09:27.791581 10525 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:27.791707 10525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:27.792006 10525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:27.794834 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "ee90d6c500e04cb0b1b07e4223e38fb8"
format_stamp: "Formatted at 2026-08-11 12:09:27 on dist-test-slave-nj21"
server_key: "3f0ecb9659bb37e9b2c1a8bfb861f335"
server_key_iv: "0f4d866bdbbaab89ed364c9882d61063"
server_key_version: "encryptionkey@0"
I20260811 12:09:27.795571 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "ee90d6c500e04cb0b1b07e4223e38fb8"
format_stamp: "Formatted at 2026-08-11 12:09:27 on dist-test-slave-nj21"
server_key: "3f0ecb9659bb37e9b2c1a8bfb861f335"
server_key_iv: "0f4d866bdbbaab89ed364c9882d61063"
server_key_version: "encryptionkey@0"
I20260811 12:09:27.800449 10525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:09:27.803916 10553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:27.805326 10525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:09:27.805469 10525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "ee90d6c500e04cb0b1b07e4223e38fb8"
format_stamp: "Formatted at 2026-08-11 12:09:27 on dist-test-slave-nj21"
server_key: "3f0ecb9659bb37e9b2c1a8bfb861f335"
server_key_iv: "0f4d866bdbbaab89ed364c9882d61063"
server_key_version: "encryptionkey@0"
I20260811 12:09:27.805627 10525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:27.822741 10525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:27.826913 10525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:27.827139 10525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:27.844425 10525 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39725
I20260811 12:09:27.844467 10643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39725 every 8 connection(s)
I20260811 12:09:27.845721 10525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:27.849336 10644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:27.853750 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10525
I20260811 12:09:27.853987 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:27.854796 4291 external_mini_cluster.cc:1428] Setting key 1524e1bc73911dc398eb8295924bd91f
I20260811 12:09:27.860824 10644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8: Bootstrap starting.
I20260811 12:09:27.863914 10644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:27.864878 10644 log.cc:824] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:27.867560 10644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8: No bootstrap required, opened a new log
Aug 11 12:09:27 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:27.875270 10644 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } }
I20260811 12:09:27.875633 10644 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:27.875765 10644 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee90d6c500e04cb0b1b07e4223e38fb8, State: Initialized, Role: FOLLOWER
I20260811 12:09:27.876370 10649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:27.856907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57926 (local address 127.4.48.254:39725)
0811 12:09:27.857702 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:27.857714 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:27.857766 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:09:27.859602 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:27.859626 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:27.860014 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:27.860379 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:27.861876 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:27.863195 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:27.864635 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:27.865388 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:27.865590 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:27.869265 (+ 3675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:27.869273 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:27.869288 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:27.869330 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:27.872436 (+ 3106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:27.873087 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:27.873097 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:27.873102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:27.873264 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:27.873634 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:27.873638 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:27.873640 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:27.874186 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:27.874474 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:27.874917 (+ 443us) server_negotiation.cc:299] Negotiation successful
0811 12:09:27.875290 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":352,"threads_started":1}
I20260811 12:09:27.876341 10644 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } }
I20260811 12:09:27.876657 10644 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:27.876811 10644 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:27.877003 10644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:27.879501 10644 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } }
I20260811 12:09:27.879961 10644 leader_election.cc:302] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee90d6c500e04cb0b1b07e4223e38fb8; no voters:
I20260811 12:09:27.880442 10644 leader_election.cc:288] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:27.880559 10653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:27.881191 10653 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [term 1 LEADER]: Becoming Leader. State: Replica: ee90d6c500e04cb0b1b07e4223e38fb8, State: Running, Role: LEADER
I20260811 12:09:27.881713 10653 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } }
I20260811 12:09:27.882257 10644 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:27.884632 10654 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } } }
I20260811 12:09:27.884572 10657 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee90d6c500e04cb0b1b07e4223e38fb8. Latest consensus state: current_term: 1 leader_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee90d6c500e04cb0b1b07e4223e38fb8" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39725 } } }
I20260811 12:09:27.885001 10654 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:27.885036 10657 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:27.886749 10662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:27.891577 10662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:27.900166 10662 catalog_manager.cc:1347] Generated new cluster ID: 51be11e45bef46589f54b5f719d787ef
I20260811 12:09:27.900264 10662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:27.913997 10662 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:27.915807 10662 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:27.924494 10662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee90d6c500e04cb0b1b07e4223e38fb8: Generated new TSK 0
I20260811 12:09:27.925256 10662 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:27.985548 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:28.326014 10684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:28.326284 10684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:28.326416 10684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:28.333392 10684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:28.333490 10684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:28.333643 10684 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:28.342828 10684 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:28.344118 10684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:28.345404 10684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:28.357725 10696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:28.357679 10697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:28.358034 10699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:28.358739 10684 server_base.cc:1034] running on GCE node
I20260811 12:09:28.359347 10684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:28.360260 10684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:28.361456 10684 hybrid_clock.cc:648] HybridClock initialized: now 1786450168361410 us; error 39 us; skew 500 ppm
I20260811 12:09:28.361742 10684 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:28 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:28.364717 10684 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:28.366032 10684 webserver.cc:460] Webserver started at http://127.4.48.193:40437/ using document root <none> and password file <none>
I20260811 12:09:28.366706 10684 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:28.366823 10684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:28.367132 10684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:28.369944 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "16ea31f193a44b0d831d307fec39b122"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "7c2e6fdcb8b54c05240d2dc36dd4c7f4"
server_key_iv: "11445e83b5f633845162e5a8aacf947d"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.370661 10684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "16ea31f193a44b0d831d307fec39b122"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "7c2e6fdcb8b54c05240d2dc36dd4c7f4"
server_key_iv: "11445e83b5f633845162e5a8aacf947d"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.375381 10684 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 12:09:28.378754 10708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.380028 10684 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 12:09:28.380173 10684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "16ea31f193a44b0d831d307fec39b122"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "7c2e6fdcb8b54c05240d2dc36dd4c7f4"
server_key_iv: "11445e83b5f633845162e5a8aacf947d"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.380342 10684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:28.398502 10684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:28.402431 10684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:28.402679 10684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:28.403550 10684 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:28.404832 10684 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:28.404917 10684 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.405035 10684 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:28.405084 10684 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.434707 10684 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:33663
I20260811 12:09:28.434753 10886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:33663 every 8 connection(s)
I20260811 12:09:28.435977 10684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:28.437650 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10684
I20260811 12:09:28.437850 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:28.438699 4291 external_mini_cluster.cc:1428] Setting key 560445f6929f662f0e2707e947feedde
Aug 11 12:09:28 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:28.457890 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:28.438486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:34231 (local address 127.4.48.254:39725)
0811 12:09:28.438889 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:28.438894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:28.440217 (+ 1323us) server_negotiation.cc:407] Connection header received
0811 12:09:28.442480 (+ 2263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:28.442484 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:28.442557 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:28.442646 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:28.444547 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.445240 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.447307 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.447791 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.447902 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:28.451829 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:28.451834 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:28.451837 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:28.451870 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:28.454371 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:28.455351 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:28.455356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:28.455358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:28.455487 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:28.456078 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:28.456082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:28.456084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:28.456348 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:28.456458 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:28.457448 (+ 990us) server_negotiation.cc:299] Negotiation successful
0811 12:09:28.457619 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":201,"threads_started":1}
I20260811 12:09:28.459259 10893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:28.439158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39725 (local address 127.4.48.193:34231)
0811 12:09:28.440051 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:28.440096 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:28.442187 (+ 2091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:28.442916 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:28.442927 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:28.443636 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:28.444376 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:28.444401 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.445413 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.445418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:28.447108 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:28.447123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.448033 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.448040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:28.448314 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:28.448978 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:28.449013 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:28.451585 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:28.454507 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:28.454515 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:28.454548 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:28.455209 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:28.455620 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:28.455625 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:28.455629 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:28.455965 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:28.456480 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:28.456488 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:28.456948 (+ 460us) client_negotiation.cc:769] Sending connection context
0811 12:09:28.457329 (+ 381us) client_negotiation.cc:240] Negotiation successful
0811 12:09:28.457858 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":347,"threads_started":1}
I20260811 12:09:28.460845 10888 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39725
I20260811 12:09:28.461126 10888 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:28.461814 10888 heartbeater.cc:511] Master 127.4.48.254:39725 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
I20260811 12:09:28.463914 10580 ts_manager.cc:194] Registered new tserver with Master: 16ea31f193a44b0d831d307fec39b122 (127.4.48.193:33663)
I20260811 12:09:28.465458 10580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:34231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:28.488448 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:28.805943 10897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:28.806198 10897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:28.806334 10897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:28.813382 10897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:28.813490 10897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:28.813637 10897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:28.822976 10897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:28.824275 10897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:28.825570 10897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:28.837925 10906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:28.838127 10907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:28.838399 10910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:28.839174 10897 server_base.cc:1034] running on GCE node
I20260811 12:09:28.839785 10897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:28.840674 10897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:28.841872 10897 hybrid_clock.cc:648] HybridClock initialized: now 1786450168841826 us; error 33 us; skew 500 ppm
I20260811 12:09:28.842171 10897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:28 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:28.845695 10897 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:28.847262 10897 webserver.cc:460] Webserver started at http://127.4.48.194:46423/ using document root <none> and password file <none>
I20260811 12:09:28.848076 10897 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:28.848199 10897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:28.848523 10897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:28.852252 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "d8a7990318bc4b33b201a6287d98b453"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "4045ab62f32b4f656cd929f5494d26b3"
server_key_iv: "f67e723e52cb1dbdf33d855d80240039"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.853202 10897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "d8a7990318bc4b33b201a6287d98b453"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "4045ab62f32b4f656cd929f5494d26b3"
server_key_iv: "f67e723e52cb1dbdf33d855d80240039"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.859114 10897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:09:28.863350 10919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.864780 10897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:28.864946 10897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "d8a7990318bc4b33b201a6287d98b453"
format_stamp: "Formatted at 2026-08-11 12:09:28 on dist-test-slave-nj21"
server_key: "4045ab62f32b4f656cd929f5494d26b3"
server_key_iv: "f67e723e52cb1dbdf33d855d80240039"
server_key_version: "encryptionkey@0"
I20260811 12:09:28.865144 10897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:28.877697 10897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:28.881556 10897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:28.881835 10897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:28.882753 10897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:28.884173 10897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:28.884264 10897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.884390 10897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:28.884449 10897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:28.913674 10897 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:34125
I20260811 12:09:28.913708 11100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:34125 every 8 connection(s)
I20260811 12:09:28.915047 10897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:28.915629 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10897
I20260811 12:09:28.915921 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:28.916541 4291 external_mini_cluster.cc:1428] Setting key 6a6f8148d901654f46f303df63670c99
Aug 11 12:09:28 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450168, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:28.938233 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:28.918298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:36505 (local address 127.4.48.254:39725)
0811 12:09:28.918503 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:28.918507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:28.919966 (+ 1459us) server_negotiation.cc:407] Connection header received
0811 12:09:28.922544 (+ 2578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:28.922548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:28.922617 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:28.922697 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:28.924767 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.925518 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.927658 (+ 2140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.928101 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.928197 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:28.932087 (+ 3890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:28.932093 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:28.932097 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:28.932133 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:28.934685 (+ 2552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:28.935577 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:28.935582 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:28.935584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:28.935825 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:28.936278 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:28.936281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:28.936283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:28.936563 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:28.936758 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:28.937623 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 12:09:28.937875 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 12:09:28.939079 11108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:28.918893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39725 (local address 127.4.48.194:36505)
0811 12:09:28.919798 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:28.919855 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:28.922243 (+ 2388us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:28.922979 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:28.922991 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:28.923741 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:28.924547 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:28.924576 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.925701 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.925707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:28.927514 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:28.927537 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:28.928332 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:28.928340 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:28.928618 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:28.929373 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:28.929397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:28.931837 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:28.934834 (+ 2997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:28.934841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:28.934855 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:28.935447 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:28.935941 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:28.935946 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:28.935950 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:28.936177 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:28.936706 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:28.936714 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:28.937111 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 12:09:28.937439 (+ 328us) client_negotiation.cc:240] Negotiation successful
0811 12:09:28.937976 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":324,"threads_started":1}
I20260811 12:09:28.940587 11101 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39725
I20260811 12:09:28.940944 11101 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
I20260811 12:09:28.941848 11101 heartbeater.cc:511] Master 127.4.48.254:39725 requested a full tablet report, sending...
I20260811 12:09:28.943259 10580 ts_manager.cc:194] Registered new tserver with Master: d8a7990318bc4b33b201a6287d98b453 (127.4.48.194:34125)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
I20260811 12:09:28.944123 10580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:36505
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:28.966943 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40465
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:29.286615 11113 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:29.286877 11113 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:29.287005 11113 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:29.294013 11113 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:29.294109 11113 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:29.294281 11113 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:29.303483 11113 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40465
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39725
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:29.304796 11113 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:29.306021 11113 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:29.318315 11127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:29.318373 11123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:29.318394 11124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:29.319335 11113 server_base.cc:1034] running on GCE node
I20260811 12:09:29.319957 11113 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:29.320848 11113 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:29.322052 11113 hybrid_clock.cc:648] HybridClock initialized: now 1786450169322004 us; error 45 us; skew 500 ppm
I20260811 12:09:29.322330 11113 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:29.325362 11113 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:29.326671 11113 webserver.cc:460] Webserver started at http://127.4.48.195:39939/ using document root <none> and password file <none>
I20260811 12:09:29.327298 11113 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:29.327415 11113 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:29.327744 11113 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:29.330543 11113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "2ee2063a01bb441cb8087965e34703f6"
format_stamp: "Formatted at 2026-08-11 12:09:29 on dist-test-slave-nj21"
server_key: "7e853d7b52ad80fb443afcfa77a60680"
server_key_iv: "09f26d7317ec85d7dab3ee0a8f71fdd9"
server_key_version: "encryptionkey@0"
I20260811 12:09:29.331292 11113 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "2ee2063a01bb441cb8087965e34703f6"
format_stamp: "Formatted at 2026-08-11 12:09:29 on dist-test-slave-nj21"
server_key: "7e853d7b52ad80fb443afcfa77a60680"
server_key_iv: "09f26d7317ec85d7dab3ee0a8f71fdd9"
server_key_version: "encryptionkey@0"
I20260811 12:09:29.336208 11113 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 12:09:29.339381 11136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:29.340662 11113 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:29.340817 11113 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "2ee2063a01bb441cb8087965e34703f6"
format_stamp: "Formatted at 2026-08-11 12:09:29 on dist-test-slave-nj21"
server_key: "7e853d7b52ad80fb443afcfa77a60680"
server_key_iv: "09f26d7317ec85d7dab3ee0a8f71fdd9"
server_key_version: "encryptionkey@0"
I20260811 12:09:29.341006 11113 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:29.352070 11113 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:29.355624 11113 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:29.355901 11113 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:29.356736 11113 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:29.357961 11113 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:29.358037 11113 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:29.358155 11113 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:29.358207 11113 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:29.388677 11113 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:33641
I20260811 12:09:29.388718 11325 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:33641 every 8 connection(s)
I20260811 12:09:29.389930 11113 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:29.395972 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 11113
I20260811 12:09:29.396189 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:29.396732 4291 external_mini_cluster.cc:1428] Setting key 54af17517887aad16e10d6d05d8c2caa
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:29.408950 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.392374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:58753 (local address 127.4.48.254:39725)
0811 12:09:29.392622 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.392629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.393705 (+ 1076us) server_negotiation.cc:407] Connection header received
0811 12:09:29.395466 (+ 1761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.395471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.395541 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.395627 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.397641 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.398591 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.400149 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.400554 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.400667 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.403848 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.403854 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.403858 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.403901 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.406354 (+ 2453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.406998 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.407002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.407004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.407125 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.407447 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.407450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.407452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.407736 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.407843 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.408466 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.408694 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:09:29.409864 11329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.392870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39725 (local address 127.4.48.195:58753)
0811 12:09:29.393553 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:29.393593 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:29.395171 (+ 1578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:29.395965 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:29.395976 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:29.396704 (+ 728us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:29.397459 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:29.397481 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.398743 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.398747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:29.400016 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:29.400028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.400775 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.400783 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:29.401010 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.401581 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:29.401602 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:29.403633 (+ 2031us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:29.406475 (+ 2842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:29.406482 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:29.406494 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:29.406907 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:29.407199 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:29.407208 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:29.407210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:29.407367 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:29.407839 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:29.407844 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:29.408149 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 12:09:29.408376 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:09:29.408792 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":251,"threads_started":1}
I20260811 12:09:29.411303 11327 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39725
I20260811 12:09:29.411644 11327 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:29.412447 11327 heartbeater.cc:511] Master 127.4.48.254:39725 requested a full tablet report, sending...
I20260811 12:09:29.413779 10579 ts_manager.cc:194] Registered new tserver with Master: 2ee2063a01bb441cb8087965e34703f6 (127.4.48.195:33641)
I20260811 12:09:29.414597 10579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:58753
I20260811 12:09:29.424156 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
I20260811 12:09:29.468302 10888 heartbeater.cc:503] Master 127.4.48.254:39725 was elected leader, sending a full tablet report...
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:29.475069 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.461091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.4.48.193:33663)
0811 12:09:29.461690 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.461698 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.461725 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:09:29.461816 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.461823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.462020 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.462150 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.463386 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.464417 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.465595 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.466157 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.466253 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.470104 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.470116 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.470133 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.470172 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.472733 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.473345 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.473350 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.473353 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.473474 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.473798 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.473801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.473803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.474105 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.474330 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.474613 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.474797 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":329,"threads_started":1}
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:29.491323 10892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.478946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57940 (local address 127.4.48.254:39725)
0811 12:09:29.479168 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.479174 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.479317 (+ 143us) server_negotiation.cc:407] Connection header received
0811 12:09:29.479434 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.479438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.479504 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.479588 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.480795 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.481509 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.482710 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.483130 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.483221 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.486731 (+ 3510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.486758 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.486761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.486794 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.489220 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.489794 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.489797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.489799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.489914 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.490189 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.490192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.490194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.490484 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.490658 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.490876 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.491032 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 12:09:29.491882 10579 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57940
I20260811 12:09:29.492805 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10684
I20260811 12:09:29.504627 11336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.493507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46992 (local address 127.4.48.193:33663)
0811 12:09:29.493696 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.493701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.493906 (+ 205us) server_negotiation.cc:407] Connection header received
0811 12:09:29.494017 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.494020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.494078 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.494147 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.495192 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.495884 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.497109 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.497502 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.497590 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.500160 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.500164 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.500168 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.500196 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.502581 (+ 2385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.503154 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.503158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.503160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.503263 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.503545 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.503548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.503550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.503798 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.503966 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.504208 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.504354 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:09:29.505219 10882 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:29.585778 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10684
I20260811 12:09:29.617213 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10897
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:29.632552 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.618245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53174 (local address 127.4.48.194:34125)
0811 12:09:29.618802 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.618809 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.618836 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:09:29.618899 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.618906 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.619118 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.619251 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.620585 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.621621 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.622889 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.623416 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.623517 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.627543 (+ 4026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.627555 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.627569 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.627602 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.630259 (+ 2657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.630885 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.630891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.630893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.631004 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.631306 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.631308 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.631310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.631614 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.631895 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.632074 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.632264 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":281,"threads_started":1}
I20260811 12:09:29.633765 11093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:29.718791 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10897
I20260811 12:09:29.746161 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 11113
Aug 11 12:09:29 dist-test-slave-nj21 krb5kdc[10501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:29.761945 11348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:29.747179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.4.48.195:33641)
0811 12:09:29.747826 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:29.747833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:29.747875 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:09:29.747937 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:29.747944 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:29.748150 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:29.748301 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:29.749582 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.750617 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.751928 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:29.752472 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:29.752586 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:29.756722 (+ 4136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:29.756734 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:29.756750 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:29.756782 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:29.759237 (+ 2455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.759936 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.759943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.759946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.760066 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:29.760417 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:29.760421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:29.760423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:29.760756 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:29.761100 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:29.761357 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 12:09:29.761614 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":316,"threads_started":1}
I20260811 12:09:29.763273 11318 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:29.844274 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 11113
I20260811 12:09:29.865753 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10525
I20260811 12:09:29.866881 10631 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:29.968658 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10525
2026-08-11T12:09:29Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (3970 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:30 dist-test-slave-nj21 krb5kdc[11360](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:30 dist-test-slave-nj21 krb5kdc[11360](info): set up 2 sockets
Aug 11 12:09:30 dist-test-slave-nj21 krb5kdc[11360](info): commencing operation
krb5kdc: starting...
W20260811 12:09:31.197683 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:31 dist-test-slave-nj21 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:31.401337 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43951
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:43951
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:31.727873 11379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:31.728170 11379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:31.728277 11379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:31.735146 11379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:31.735239 11379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:31.735289 11379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:31.735344 11379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:31.735391 11379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:31.744920 11379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:43951
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:31.746088 11379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:31.747319 11379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:31.758193 11389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:31.758328 11387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:31.758291 11385 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:31.759382 11379 server_base.cc:1034] running on GCE node
I20260811 12:09:31.760141 11379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:31.761399 11379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:31.762606 11379 hybrid_clock.cc:648] HybridClock initialized: now 1786450171762554 us; error 46 us; skew 500 ppm
I20260811 12:09:31.762949 11379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:31 dist-test-slave-nj21 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:31.770713 11379 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:31.773021 11379 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:31.773126 11379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:31.773442 11379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:31.776346 11379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "8f87535c7ad14a1aa29285e078053a8c"
format_stamp: "Formatted at 2026-08-11 12:09:31 on dist-test-slave-nj21"
server_key: "4dcbea83978fe3347afdc7f7f2c201f7"
server_key_iv: "3f3c8623bfc193ccb770242222b3ce87"
server_key_version: "encryptionkey@0"
I20260811 12:09:31.777098 11379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "8f87535c7ad14a1aa29285e078053a8c"
format_stamp: "Formatted at 2026-08-11 12:09:31 on dist-test-slave-nj21"
server_key: "4dcbea83978fe3347afdc7f7f2c201f7"
server_key_iv: "3f3c8623bfc193ccb770242222b3ce87"
server_key_version: "encryptionkey@0"
I20260811 12:09:31.781822 11379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 12:09:31.784996 11398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:31.786136 11379 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:09:31.786305 11379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "8f87535c7ad14a1aa29285e078053a8c"
format_stamp: "Formatted at 2026-08-11 12:09:31 on dist-test-slave-nj21"
server_key: "4dcbea83978fe3347afdc7f7f2c201f7"
server_key_iv: "3f3c8623bfc193ccb770242222b3ce87"
server_key_version: "encryptionkey@0"
I20260811 12:09:31.786501 11379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:31.806406 11379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:31.810240 11379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:31.810487 11379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:31.828016 11379 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:43951
I20260811 12:09:31.827994 11490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:43951 every 8 connection(s)
I20260811 12:09:31.828881 11379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:31.829944 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 11379
I20260811 12:09:31.830199 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:31.831012 4291 external_mini_cluster.cc:1428] Setting key 67e1c0a9bda5c91e50d7edddd8e82bdd
I20260811 12:09:31.833541 11491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:31.843822 11491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: Bootstrap starting.
Aug 11 12:09:31 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:31.847218 11491 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:31.848378 11491 log.cc:824] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:31.851660 11491 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: No bootstrap required, opened a new log
I20260811 12:09:31.852803 11494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:31.833121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33890 (local address 127.4.48.254:43951)
0811 12:09:31.833919 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:31.833931 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:31.833970 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:09:31.835564 (+ 1594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:31.835586 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:31.836000 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:31.836370 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:31.837888 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:31.839150 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:31.840416 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:31.841109 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:31.841295 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:31.845320 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:31.845328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:31.845342 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:31.845383 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:31.848304 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:31.849071 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:31.849077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:31.849083 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:31.849248 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:31.849684 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:31.849689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:31.849692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:31.850427 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:31.850737 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:31.851057 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 12:09:31.851481 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":361,"threads_started":1}
I20260811 12:09:31.859764 11491 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } }
I20260811 12:09:31.860095 11491 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:31.860199 11491 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f87535c7ad14a1aa29285e078053a8c, State: Initialized, Role: FOLLOWER
I20260811 12:09:31.860725 11491 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } }
I20260811 12:09:31.860945 11491 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:31.861079 11491 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:31.861229 11491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:31.863548 11491 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } }
I20260811 12:09:31.864056 11491 leader_election.cc:302] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f87535c7ad14a1aa29285e078053a8c; no voters:
I20260811 12:09:31.864619 11491 leader_election.cc:288] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:31.864738 11497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:31.865401 11497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [term 1 LEADER]: Becoming Leader. State: Replica: 8f87535c7ad14a1aa29285e078053a8c, State: Running, Role: LEADER
I20260811 12:09:31.865923 11497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } }
I20260811 12:09:31.866551 11491 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:31.868702 11500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f87535c7ad14a1aa29285e078053a8c. Latest consensus state: current_term: 1 leader_uuid: "8f87535c7ad14a1aa29285e078053a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } } }
I20260811 12:09:31.869088 11500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:31.869804 11498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f87535c7ad14a1aa29285e078053a8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f87535c7ad14a1aa29285e078053a8c" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43951 } } }
I20260811 12:09:31.870170 11498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:31.871392 11512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:31.875612 11512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:31.883265 11512 catalog_manager.cc:1347] Generated new cluster ID: 0f04ad219a624c7ca8f443ff1705e4ba
I20260811 12:09:31.883355 11512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:31.897831 11512 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:31.899710 11512 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:31.918310 11512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: Generated new TSK 0
I20260811 12:09:31.919384 11512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:31.971014 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:32.291566 11532 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:32.291886 11532 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:32.292034 11532 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:32.299081 11532 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:32.299175 11532 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:32.299331 11532 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:32.309252 11532 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:32.310523 11532 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:32.311837 11532 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:32.323900 11539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:32.323945 11540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:32.324079 11543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:32.324918 11532 server_base.cc:1034] running on GCE node
I20260811 12:09:32.325421 11532 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:32.326256 11532 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:32.327446 11532 hybrid_clock.cc:648] HybridClock initialized: now 1786450172327407 us; error 26 us; skew 500 ppm
I20260811 12:09:32.327749 11532 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:32 dist-test-slave-nj21 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450172, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:32.330682 11532 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:32.331826 11532 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:32.331923 11532 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:32.332185 11532 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:32.334941 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "3bf078fa4a9b45e8a38e2eed4392f007"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "8db0082597c81a2b98708fb06b3c6d75"
server_key_iv: "3995452bac5a809a30ad66869a622b2a"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.335711 11532 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "3bf078fa4a9b45e8a38e2eed4392f007"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "8db0082597c81a2b98708fb06b3c6d75"
server_key_iv: "3995452bac5a809a30ad66869a622b2a"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.340364 11532 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:09:32.343532 11557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.344769 11532 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:09:32.344905 11532 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "3bf078fa4a9b45e8a38e2eed4392f007"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "8db0082597c81a2b98708fb06b3c6d75"
server_key_iv: "3995452bac5a809a30ad66869a622b2a"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.345077 11532 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:32.356606 11532 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:32.360669 11532 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:32.360906 11532 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:32.361795 11532 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:32.363057 11532 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:32.363142 11532 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.363219 11532 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:32.363271 11532 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.393160 11532 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:33751
I20260811 12:09:32.393224 11723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:33751 every 8 connection(s)
I20260811 12:09:32.394001 11532 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:32.398370 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 11532
I20260811 12:09:32.398586 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:32.399364 4291 external_mini_cluster.cc:1428] Setting key a79a220fbde23001b25aa59a4116475f
Aug 11 12:09:32 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450172, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:32.414438 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:32.396309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:39625 (local address 127.4.48.254:43951)
0811 12:09:32.396806 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:32.396813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:32.397703 (+ 890us) server_negotiation.cc:407] Connection header received
0811 12:09:32.400119 (+ 2416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:32.400126 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:32.400212 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:32.400325 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:32.402006 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.402709 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.404810 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.405186 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.405274 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:32.408959 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:32.408964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:32.408967 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:32.409000 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:32.411470 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:32.412216 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:32.412219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:32.412222 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:32.412345 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:32.412746 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:32.412750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:32.412752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:32.413016 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:32.413148 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:32.413836 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 12:09:32.414057 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":236,"threads_started":1}
I20260811 12:09:32.415369 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:32.396842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43951 (local address 127.4.48.193:39625)
0811 12:09:32.397546 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:32.397591 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:32.399804 (+ 2213us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:32.400561 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:32.400569 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:32.401095 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:32.401837 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:32.401863 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.402871 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.402877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:32.404685 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:32.404699 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.405404 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.405411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:32.405656 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:32.406304 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:32.406337 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:32.408733 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:32.411600 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:32.411609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:32.411625 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:32.412094 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:32.412449 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:32.412454 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:32.412458 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:32.412634 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:32.413140 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:32.413147 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:32.413474 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 12:09:32.413716 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 12:09:32.414216 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":270,"threads_started":1}
I20260811 12:09:32.417021 11724 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43951
I20260811 12:09:32.417299 11724 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:32.417996 11724 heartbeater.cc:511] Master 127.4.48.254:43951 requested a full tablet report, sending...
I20260811 12:09:32.420132 11426 ts_manager.cc:194] Registered new tserver with Master: 3bf078fa4a9b45e8a38e2eed4392f007 (127.4.48.193:33751)
I20260811 12:09:32.421646 11426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:39625
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:32.448484 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:32.790275 11732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:32.790537 11732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:32.790673 11732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:32.797814 11732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:32.797917 11732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:32.798059 11732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:32.807508 11732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:32.808815 11732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:32.810137 11732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:32.822122 11738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:32.822213 11739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:32.822228 11742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:32.822942 11732 server_base.cc:1034] running on GCE node
I20260811 12:09:32.823488 11732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:32.824416 11732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:32.825613 11732 hybrid_clock.cc:648] HybridClock initialized: now 1786450172825564 us; error 48 us; skew 500 ppm
I20260811 12:09:32.825927 11732 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:32 dist-test-slave-nj21 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450172, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:32.828998 11732 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:32.830174 11732 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:32.830261 11732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:32.830562 11732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:32.833457 11732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "e4fb89319e8d4854893ceacf09b482be"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "f40cb05541358066a487a0b7c389d996"
server_key_iv: "aa574832526b5f950a991fe76a9282ed"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.834234 11732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "e4fb89319e8d4854893ceacf09b482be"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "f40cb05541358066a487a0b7c389d996"
server_key_iv: "aa574832526b5f950a991fe76a9282ed"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.839016 11732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 12:09:32.842329 11749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.843617 11732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:32.843837 11732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "e4fb89319e8d4854893ceacf09b482be"
format_stamp: "Formatted at 2026-08-11 12:09:32 on dist-test-slave-nj21"
server_key: "f40cb05541358066a487a0b7c389d996"
server_key_iv: "aa574832526b5f950a991fe76a9282ed"
server_key_version: "encryptionkey@0"
I20260811 12:09:32.844024 11732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:32.871613 11732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:32.875326 11732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:32.875564 11732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:32.876492 11732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:32.877808 11732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:32.877913 11732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.878005 11732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:32.878068 11732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:32.908697 11732 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:42251
I20260811 12:09:32.908766 11913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:42251 every 8 connection(s)
I20260811 12:09:32.909565 11732 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:32.918773 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 11732
I20260811 12:09:32.918962 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:32.919552 4291 external_mini_cluster.cc:1428] Setting key de269a7f6b1faa4c8ead8a9de9a3f3bc
Aug 11 12:09:32 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450172, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:32.928957 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:32.911950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:50009 (local address 127.4.48.254:43951)
0811 12:09:32.912122 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:32.912127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:32.913207 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 12:09:32.915007 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:32.915013 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:32.915095 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:32.915186 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:32.916702 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.917431 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.918947 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.919558 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.919706 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:32.922943 (+ 3237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:32.922949 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:32.922954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:32.922992 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:32.925851 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:32.926529 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:32.926533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:32.926536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:32.926705 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:32.927081 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:32.927085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:32.927087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:32.927403 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:32.927531 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:32.928304 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 12:09:32.928632 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:09:32.929914 11917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:32.912370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43951 (local address 127.4.48.194:50009)
0811 12:09:32.913050 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:32.913094 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:32.914725 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:32.915424 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:32.915432 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:32.915995 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:32.916529 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:32.916545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.917581 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.917585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:32.918786 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:32.918799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:32.919836 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:32.919842 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:32.920024 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:32.920621 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:32.920648 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:32.922726 (+ 2078us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:32.925982 (+ 3256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:32.925988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:32.925999 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:32.926403 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:32.926804 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:32.926808 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:32.926810 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:32.926973 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:32.927508 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:32.927513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:32.927841 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 12:09:32.928084 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 12:09:32.928580 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":272,"threads_started":1}
I20260811 12:09:32.931576 11915 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43951
I20260811 12:09:32.931880 11915 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:32.932494 11915 heartbeater.cc:511] Master 127.4.48.254:43951 requested a full tablet report, sending...
I20260811 12:09:32.933931 11426 ts_manager.cc:194] Registered new tserver with Master: e4fb89319e8d4854893ceacf09b482be (127.4.48.194:42251)
I20260811 12:09:32.934679 11426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:50009
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:32.966917 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:33.282677 11922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:33.282927 11922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:33.283044 11922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:33.289981 11922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:33.290089 11922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:33.290251 11922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:33.299546 11922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43951
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:33.300760 11922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:33.301992 11922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:33.313979 11940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:33.314066 11932 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:33.314044 11936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:33.315045 11922 server_base.cc:1034] running on GCE node
I20260811 12:09:33.315627 11922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:33.316550 11922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:33.317759 11922 hybrid_clock.cc:648] HybridClock initialized: now 1786450173317689 us; error 53 us; skew 500 ppm
I20260811 12:09:33.318050 11922 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:33 dist-test-slave-nj21 krb5kdc[11360](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:33.321226 11922 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:33.322335 11922 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:33.322439 11922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:33.322731 11922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:33.325595 11922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "335804479f8a4139bf1a2257ce1fb71e"
format_stamp: "Formatted at 2026-08-11 12:09:33 on dist-test-slave-nj21"
server_key: "82118d6535ec36eb1060908d1f34b3ad"
server_key_iv: "9c4273bc3834a45bb8dbe877d85e2e42"
server_key_version: "encryptionkey@0"
I20260811 12:09:33.326388 11922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "335804479f8a4139bf1a2257ce1fb71e"
format_stamp: "Formatted at 2026-08-11 12:09:33 on dist-test-slave-nj21"
server_key: "82118d6535ec36eb1060908d1f34b3ad"
server_key_iv: "9c4273bc3834a45bb8dbe877d85e2e42"
server_key_version: "encryptionkey@0"
I20260811 12:09:33.331288 11922 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260811 12:09:33.335044 11951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:33.336315 11922 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:09:33.336460 11922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "335804479f8a4139bf1a2257ce1fb71e"
format_stamp: "Formatted at 2026-08-11 12:09:33 on dist-test-slave-nj21"
server_key: "82118d6535ec36eb1060908d1f34b3ad"
server_key_iv: "9c4273bc3834a45bb8dbe877d85e2e42"
server_key_version: "encryptionkey@0"
I20260811 12:09:33.336664 11922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:33.355330 11922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:33.359108 11922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:33.359345 11922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:33.360292 11922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:33.361580 11922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:33.361675 11922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:33.361781 11922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:33.361855 11922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:33.392607 11922 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39679
I20260811 12:09:33.392624 12131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39679 every 8 connection(s)
I20260811 12:09:33.393720 11922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:33.394578 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 11922
I20260811 12:09:33.394778 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:33.395352 4291 external_mini_cluster.cc:1428] Setting key a83ba74f1fc61cc13a4abaa7351e9987
Aug 11 12:09:33 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450173, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:33.412834 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.396663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:59165 (local address 127.4.48.254:43951)
0811 12:09:33.396841 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.396848 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.398115 (+ 1267us) server_negotiation.cc:407] Connection header received
0811 12:09:33.400141 (+ 2026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.400146 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.400235 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.400325 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:33.401656 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.402372 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.403921 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.404319 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.404410 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.407483 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:33.407488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:33.407491 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:33.407524 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:33.410141 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.410940 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.410945 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.410948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.411074 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.411451 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.411454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.411456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.411752 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:33.411864 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.412443 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.412594 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:09:33.413759 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.397148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43951 (local address 127.4.48.195:59165)
0811 12:09:33.397954 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.398007 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.399899 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.400494 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.400501 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.401001 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:33.401531 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.401548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.402489 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.402492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.403770 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.403783 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.404480 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.404484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.404655 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.405218 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:33.405240 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:33.407257 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:33.410270 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:33.410279 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:33.410294 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:33.410806 (+ 512us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:33.411180 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:33.411185 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:33.411189 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:33.411372 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:33.411875 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:33.411882 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:33.412164 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.412386 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.412735 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":340,"threads_started":1}
I20260811 12:09:33.415010 12132 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43951
I20260811 12:09:33.415272 12132 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:33.415915 12132 heartbeater.cc:511] Master 127.4.48.254:43951 requested a full tablet report, sending...
I20260811 12:09:33.417222 11426 ts_manager.cc:194] Registered new tserver with Master: 335804479f8a4139bf1a2257ce1fb71e (127.4.48.195:39679)
I20260811 12:09:33.418021 11426 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:59165
I20260811 12:09:33.423125 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:33.424458 11724 heartbeater.cc:503] Master 127.4.48.254:43951 was elected leader, sending a full tablet report...
I20260811 12:09:33.439347 11726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.427570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33906 (local address 127.4.48.254:43951)
0811 12:09:33.427767 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.427773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.428189 (+ 416us) server_negotiation.cc:407] Connection header received
0811 12:09:33.428342 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.428347 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.428414 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.428491 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:33.429925 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.430624 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.431872 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.432313 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.432430 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.434633 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:33.434638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:33.434641 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:33.434670 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:33.436924 (+ 2254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.437511 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.437515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.437517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.437640 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.437987 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.437992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.437995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.438298 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:33.438529 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.438856 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.439071 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:09:33.443204 11426 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:33.450107 11426 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:33.470731 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.461713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.4.48.193:33751)
0811 12:09:33.462295 (+ 582us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.462301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.462491 (+ 190us) server_negotiation.cc:407] Connection header received
0811 12:09:33.462778 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.462787 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.463040 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.463239 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:33.464195 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.465685 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.468394 (+ 2709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.469570 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.469702 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.469954 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.470236 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.470402 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":283,"threads_started":1}
I20260811 12:09:33.470777 12147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.461518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:33751 (local address 127.0.0.1:40662)
0811 12:09:33.462366 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.462406 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.462580 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.463523 (+ 943us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.463531 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.463563 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:33.464011 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.464022 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.465888 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.465893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.468206 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.468221 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.469784 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.469791 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.469993 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.470046 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.470184 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.470380 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":497,"threads_started":1}
I20260811 12:09:33.472909 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.463314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39679 (local address 127.0.0.1:33198)
0811 12:09:33.463861 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.463885 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.464002 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.466167 (+ 2165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.466172 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.466203 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:33.466600 (+ 397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.466613 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.468520 (+ 1907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.468525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.470204 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.470217 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.472087 (+ 1870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.472093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.472258 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.472292 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.472364 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.472494 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"mutex_wait_us":36,"thread_start_us":1631,"threads_started":1}
I20260811 12:09:33.473116 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.462724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:42251 (local address 127.0.0.1:34694)
0811 12:09:33.463217 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.463245 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.463344 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.464988 (+ 1644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.464993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.465019 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:33.465466 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.465480 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.467219 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.467224 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.470439 (+ 3215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.470451 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.472494 (+ 2043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.472499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.472610 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.472632 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.472702 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.472783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":1324,"thread_start_us":236,"threads_started":1}
I20260811 12:09:33.473194 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.462967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.4.48.194:42251)
0811 12:09:33.464459 (+ 1492us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.464468 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.464494 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:09:33.464586 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.464593 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.464796 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.464976 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:33.465690 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.467101 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.470613 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.472315 (+ 1702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.472443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.472629 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.472740 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.472909 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321,"thread_start_us":1224,"threads_started":1}
I20260811 12:09:33.474020 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.464709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33198 (local address 127.4.48.195:39679)
0811 12:09:33.465566 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.465573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.465601 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:09:33.465689 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.465696 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.465900 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.466132 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:33.466804 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.468371 (+ 1567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.470363 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.471852 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.473380 (+ 1528us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.473605 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.473665 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.473772 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":703,"thread_start_us":1144,"threads_started":1}
I20260811 12:09:33.476024 11622 tablet_service.cc:1467] Processing CreateTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 (DEFAULT_TABLE table=test-table [id=5bf85650b83f4775ab46a119c79ef74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:33.476769 11815 tablet_service.cc:1467] Processing CreateTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 (DEFAULT_TABLE table=test-table [id=5bf85650b83f4775ab46a119c79ef74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:33.477600 11622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33e230cd61e4dfb864bf4c3e35e2e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:33.477914 11815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33e230cd61e4dfb864bf4c3e35e2e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:33.477567 12017 tablet_service.cc:1467] Processing CreateTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 (DEFAULT_TABLE table=test-table [id=5bf85650b83f4775ab46a119c79ef74b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:33.478857 12017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f33e230cd61e4dfb864bf4c3e35e2e59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:33.488693 12158 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Bootstrap starting.
I20260811 12:09:33.491802 12160 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Bootstrap starting.
I20260811 12:09:33.491878 12158 tablet_bootstrap.cc:654] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:33.493180 12158 log.cc:824] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:33.495553 12160 tablet_bootstrap.cc:654] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:33.496837 12160 log.cc:824] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:33.497653 12161 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Bootstrap starting.
I20260811 12:09:33.500619 12161 tablet_bootstrap.cc:654] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:33.501859 12161 log.cc:824] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:33.502115 12158 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: No bootstrap required, opened a new log
I20260811 12:09:33.502422 12158 ts_tablet_manager.cc:1397] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.013s
I20260811 12:09:33.502434 12160 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: No bootstrap required, opened a new log
I20260811 12:09:33.502703 12160 ts_tablet_manager.cc:1397] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Time spent bootstrapping tablet: real 0.011s user 0.011s sys 0.000s
I20260811 12:09:33.505151 12161 tablet_bootstrap.cc:492] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: No bootstrap required, opened a new log
I20260811 12:09:33.505402 12161 ts_tablet_manager.cc:1397] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:09:33.509738 12158 raft_consensus.cc:348] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.510070 12158 raft_consensus.cc:374] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:33.510169 12158 raft_consensus.cc:729] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fb89319e8d4854893ceacf09b482be, State: Initialized, Role: FOLLOWER
I20260811 12:09:33.510722 12158 consensus_queue.cc:260] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.511811 11915 heartbeater.cc:503] Master 127.4.48.254:43951 was elected leader, sending a full tablet report...
I20260811 12:09:33.512385 12158 ts_tablet_manager.cc:1428] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 12:09:33.512643 12160 raft_consensus.cc:348] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.513105 12160 raft_consensus.cc:374] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:33.513249 12160 raft_consensus.cc:729] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bf078fa4a9b45e8a38e2eed4392f007, State: Initialized, Role: FOLLOWER
I20260811 12:09:33.513775 12160 consensus_queue.cc:260] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.515357 12160 ts_tablet_manager.cc:1428] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 12:09:33.516582 12161 raft_consensus.cc:348] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.516937 12161 raft_consensus.cc:374] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:33.517030 12161 raft_consensus.cc:729] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 335804479f8a4139bf1a2257ce1fb71e, State: Initialized, Role: FOLLOWER
I20260811 12:09:33.517525 12161 consensus_queue.cc:260] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.518538 12132 heartbeater.cc:503] Master 127.4.48.254:43951 was elected leader, sending a full tablet report...
I20260811 12:09:33.518996 12161 ts_tablet_manager.cc:1428] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Time spent starting tablet: real 0.013s user 0.009s sys 0.004s
W20260811 12:09:33.646184 11725 tablet.cc:2369] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:09:33.646785 12133 tablet.cc:2369] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:09:33.661264 11916 tablet.cc:2369] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:33.755424 12176 raft_consensus.cc:482] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:33.755709 12176 raft_consensus.cc:504] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.757454 12176 leader_election.cc:288] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bf078fa4a9b45e8a38e2eed4392f007 (127.4.48.193:33751), e4fb89319e8d4854893ceacf09b482be (127.4.48.194:42251)
I20260811 12:09:33.763769 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.757850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:38435 (local address 127.4.48.193:33751)
0811 12:09:33.758023 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.758027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.758084 (+ 57us) server_negotiation.cc:407] Connection header received
0811 12:09:33.758591 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.758597 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.758678 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.758793 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:33.759195 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.760189 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.761952 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.762907 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.763023 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.763077 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.763412 (+ 335us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.763501 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:09:33.763824 12134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.757799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:33751 (local address 127.4.48.195:38435)
0811 12:09:33.758003 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.758023 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.758150 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.758807 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.758812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.758842 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:33.759048 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.759061 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.760319 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.760324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.761792 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.761803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.763051 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.763057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.763247 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.763293 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.763393 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.763523 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260811 12:09:33.764387 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.758435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:41655 (local address 127.4.48.194:42251)
0811 12:09:33.758607 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.758612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.758956 (+ 344us) server_negotiation.cc:407] Connection header received
0811 12:09:33.759103 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.759108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.759189 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.759260 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:33.759846 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.760695 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.762518 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.763435 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.763560 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.763625 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.764034 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.764165 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:09:33.764869 12178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.758193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:42251 (local address 127.4.48.195:41655)
0811 12:09:33.758873 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:33.758887 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:33.758990 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:33.759307 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:33.759312 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:33.759337 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:33.759693 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.759705 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.760847 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.760851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.762375 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:33.762389 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.763629 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.763635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:33.763827 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.763858 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:09:33.764540 (+ 682us) client_negotiation.cc:240] Negotiation successful
0811 12:09:33.764639 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":365,"threads_started":1}
I20260811 12:09:33.764755 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33e230cd61e4dfb864bf4c3e35e2e59" candidate_uuid: "335804479f8a4139bf1a2257ce1fb71e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" is_pre_election: true
I20260811 12:09:33.765249 11655 raft_consensus.cc:2454] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 335804479f8a4139bf1a2257ce1fb71e in term 0.
I20260811 12:09:33.765729 11847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33e230cd61e4dfb864bf4c3e35e2e59" candidate_uuid: "335804479f8a4139bf1a2257ce1fb71e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fb89319e8d4854893ceacf09b482be" is_pre_election: true
I20260811 12:09:33.765888 11955 leader_election.cc:302] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 335804479f8a4139bf1a2257ce1fb71e, 3bf078fa4a9b45e8a38e2eed4392f007; no voters:
I20260811 12:09:33.766256 11847 raft_consensus.cc:2454] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 335804479f8a4139bf1a2257ce1fb71e in term 0.
I20260811 12:09:33.766294 12176 raft_consensus.cc:2781] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:33.766418 12176 raft_consensus.cc:482] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:33.766507 12176 raft_consensus.cc:3037] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:33.768778 12176 raft_consensus.cc:504] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.769316 12176 leader_election.cc:288] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [CANDIDATE]: Term 1 election: Requested vote from peers 3bf078fa4a9b45e8a38e2eed4392f007 (127.4.48.193:33751), e4fb89319e8d4854893ceacf09b482be (127.4.48.194:42251)
I20260811 12:09:33.769706 11655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33e230cd61e4dfb864bf4c3e35e2e59" candidate_uuid: "335804479f8a4139bf1a2257ce1fb71e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bf078fa4a9b45e8a38e2eed4392f007"
I20260811 12:09:33.769734 11847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f33e230cd61e4dfb864bf4c3e35e2e59" candidate_uuid: "335804479f8a4139bf1a2257ce1fb71e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fb89319e8d4854893ceacf09b482be"
I20260811 12:09:33.769872 11655 raft_consensus.cc:3037] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:33.769877 11847 raft_consensus.cc:3037] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:33.772049 11847 raft_consensus.cc:2454] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 335804479f8a4139bf1a2257ce1fb71e in term 1.
I20260811 12:09:33.772149 11655 raft_consensus.cc:2454] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 335804479f8a4139bf1a2257ce1fb71e in term 1.
I20260811 12:09:33.772397 11955 leader_election.cc:302] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 335804479f8a4139bf1a2257ce1fb71e, e4fb89319e8d4854893ceacf09b482be; no voters:
I20260811 12:09:33.772653 12176 raft_consensus.cc:2781] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:33.773063 12176 raft_consensus.cc:686] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 1 LEADER]: Becoming Leader. State: Replica: 335804479f8a4139bf1a2257ce1fb71e, State: Running, Role: LEADER
I20260811 12:09:33.773519 12176 consensus_queue.cc:237] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } }
I20260811 12:09:33.778009 11426 catalog_manager.cc:5495] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e reported cstate change: term changed from 0 to 1, leader changed from <none> to 335804479f8a4139bf1a2257ce1fb71e (127.4.48.195). New cstate: current_term: 1 leader_uuid: "335804479f8a4139bf1a2257ce1fb71e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "335804479f8a4139bf1a2257ce1fb71e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39679 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:33.837256 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.831544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33202 (local address 127.4.48.195:39679)
0811 12:09:33.831766 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.831770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.831788 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:09:33.831977 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.831981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.832068 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.832190 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:33.832591 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.833276 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.835225 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.835601 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.835722 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.836232 (+ 510us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:33.836747 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:33.836853 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.836966 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.837032 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:09:33.845155 11847 raft_consensus.cc:1261] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 1 FOLLOWER]: Refusing update from remote peer 335804479f8a4139bf1a2257ce1fb71e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:33.845208 11655 raft_consensus.cc:1261] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 1 FOLLOWER]: Refusing update from remote peer 335804479f8a4139bf1a2257ce1fb71e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:33.846073 12179 consensus_queue.cc:1035] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4fb89319e8d4854893ceacf09b482be" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 42251 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:33.846417 12176 consensus_queue.cc:1035] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bf078fa4a9b45e8a38e2eed4392f007" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 33751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:33.862316 12190 mvcc.cc:205] Tried to move back new op lower bound from 7317299912059711488 to 7317299911779528704. Current Snapshot: MvccSnapshot[applied={T|T < 7317299912059711488}]
I20260811 12:09:33.870630 11426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:33.870885 11426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:33.874426 11426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f87535c7ad14a1aa29285e078053a8c: Sending DeleteTablet for 3 replicas of tablet f33e230cd61e4dfb864bf4c3e35e2e59
I20260811 12:09:33.875483 11622 tablet_service.cc:1514] Processing DeleteTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:33 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:40662
I20260811 12:09:33.875728 12017 tablet_service.cc:1514] Processing DeleteTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:33 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:33198
I20260811 12:09:33.875952 11815 tablet_service.cc:1514] Processing DeleteTablet for tablet f33e230cd61e4dfb864bf4c3e35e2e59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:33 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:34694
I20260811 12:09:33.877705 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 11532
I20260811 12:09:33.886641 12206 tablet_replica.cc:330] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: stopping tablet replica
I20260811 12:09:33.887329 12202 tablet_replica.cc:330] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: stopping tablet replica
I20260811 12:09:33.887633 12206 raft_consensus.cc:2229] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:33.888311 12206 raft_consensus.cc:2258] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:33.888547 12202 raft_consensus.cc:2229] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:33.889032 12202 raft_consensus.cc:2258] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:33.891096 12206 ts_tablet_manager.cc:1905] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:33.892067 12202 ts_tablet_manager.cc:1905] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:33.892656 12204 tablet_replica.cc:330] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: stopping tablet replica
I20260811 12:09:33.893077 12204 raft_consensus.cc:2229] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 1 LEADER]: Raft consensus shutting down.
Aug 11 12:09:33 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:33.896341 12204 raft_consensus.cc:2258] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:33.898433 12204 ts_tablet_manager.cc:1905] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:33.900094 12206 ts_tablet_manager.cc:1918] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:33.900309 12206 log.cc:1196] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/f33e230cd61e4dfb864bf4c3e35e2e59
I20260811 12:09:33.900897 12206 ts_tablet_manager.cc:1939] T f33e230cd61e4dfb864bf4c3e35e2e59 P e4fb89319e8d4854893ceacf09b482be: Deleting consensus metadata
I20260811 12:09:33.903577 11402 catalog_manager.cc:4841] TS e4fb89319e8d4854893ceacf09b482be (127.4.48.194:42251): tablet f33e230cd61e4dfb864bf4c3e35e2e59 (table test-table [id=5bf85650b83f4775ab46a119c79ef74b]) successfully deleted
I20260811 12:09:33.904808 12202 ts_tablet_manager.cc:1918] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:33.905074 12202 log.cc:1196] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/f33e230cd61e4dfb864bf4c3e35e2e59
I20260811 12:09:33.905171 12148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:33.879374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.4.48.193:33751)
0811 12:09:33.879553 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:33.879560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:33.884482 (+ 4922us) server_negotiation.cc:407] Connection header received
0811 12:09:33.884653 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:33.884658 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:33.884763 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:33.884877 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:33.887994 (+ 3117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.889130 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.891089 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:33.891780 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:33.891878 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:33.897121 (+ 5243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:33.897154 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:33.897172 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:33.897213 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:33.900924 (+ 3711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.901783 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.901792 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.901796 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.901962 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:33.902434 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:33.902439 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:33.902442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:33.902898 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:33.903223 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:33.904489 (+ 1266us) server_negotiation.cc:299] Negotiation successful
0811 12:09:33.904703 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:09:33.905392 12204 ts_tablet_manager.cc:1918] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:33.905540 12204 log.cc:1196] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/f33e230cd61e4dfb864bf4c3e35e2e59
I20260811 12:09:33.906057 12204 ts_tablet_manager.cc:1939] T f33e230cd61e4dfb864bf4c3e35e2e59 P 335804479f8a4139bf1a2257ce1fb71e: Deleting consensus metadata
I20260811 12:09:33.906303 12202 ts_tablet_manager.cc:1939] T f33e230cd61e4dfb864bf4c3e35e2e59 P 3bf078fa4a9b45e8a38e2eed4392f007: Deleting consensus metadata
I20260811 12:09:33.906572 11718 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:33.907653 11402 catalog_manager.cc:4841] TS 335804479f8a4139bf1a2257ce1fb71e (127.4.48.195:39679): tablet f33e230cd61e4dfb864bf4c3e35e2e59 (table test-table [id=5bf85650b83f4775ab46a119c79ef74b]) successfully deleted
I20260811 12:09:34.004025 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 11532
I20260811 12:09:34.004266 11402 catalog_manager.cc:4841] TS 3bf078fa4a9b45e8a38e2eed4392f007 (127.4.48.193:33751): tablet f33e230cd61e4dfb864bf4c3e35e2e59 (table test-table [id=5bf85650b83f4775ab46a119c79ef74b]) successfully deleted
I20260811 12:09:34.036693 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 11732
Aug 11 12:09:34 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:34.051410 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:34.037449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.4.48.194:42251)
0811 12:09:34.037637 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:34.037643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:34.037663 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:09:34.037794 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:34.037800 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:34.037877 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:34.037975 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:34.039181 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:34.040059 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:34.041366 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:34.041774 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:34.041892 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:34.045902 (+ 4010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:34.045918 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:34.045934 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:34.045968 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:34.048741 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:34.049383 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:34.049391 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:34.049393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:34.049542 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:34.049895 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:34.049899 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:34.049900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:34.050280 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:34.050555 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:34.050876 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 12:09:34.051084 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:09:34.052110 11907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:34.139067 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 11732
I20260811 12:09:34.162804 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 11922
Aug 11 12:09:34 dist-test-slave-nj21 krb5kdc[11360](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450171, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:34.178433 12157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:34.163706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33206 (local address 127.4.48.195:39679)
0811 12:09:34.163893 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:34.163906 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:34.163928 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:09:34.164066 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:34.164071 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:34.164145 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:34.164229 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:34.165487 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:34.166303 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:34.167568 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:34.168146 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:34.168268 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:34.172621 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:34.172634 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:34.172651 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:34.172684 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:34.175823 (+ 3139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:34.176427 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:34.176434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:34.176436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:34.176605 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:34.176956 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:34.176960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:34.176962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:34.177324 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:34.177633 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:34.177878 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 12:09:34.178090 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:09:34.179073 12125 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:34.301878 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 11922
I20260811 12:09:34.326776 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 11379
I20260811 12:09:34.327962 11485 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:34.433399 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 11379
2026-08-11T12:09:34Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4458 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T12:09:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:34Z Disabled control of system clock
I20260811 12:09:34.509196 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:37559
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:37559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:09:34.830920 12222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:34.831192 12222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:34.831279 12222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:34.838444 12222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:34.838541 12222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:34.838587 12222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:34.838644 12222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:34.838686 12222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:34.847579 12222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:37559
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:37559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:34.848819 12222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:34.850565 12222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:34.861992 12229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:34.861925 12234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:34.861886 12230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:34.862924 12222 server_base.cc:1034] running on GCE node
I20260811 12:09:34.863782 12222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:34.865183 12222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:34.866407 12222 hybrid_clock.cc:648] HybridClock initialized: now 1786450174866361 us; error 33 us; skew 500 ppm
I20260811 12:09:34.866786 12222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:34.869046 12222 webserver.cc:460] Webserver started at http://127.4.48.254:38413/ using document root <none> and password file <none>
I20260811 12:09:34.869745 12222 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:34.869848 12222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:34.870162 12222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:34.873310 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "d009160d49574d18a5bce9042a90adeb"
format_stamp: "Formatted at 2026-08-11 12:09:34 on dist-test-slave-nj21"
server_key: "24d268e4ec7448ac366d4ec77289b378"
server_key_iv: "5d8170384fcebe491336e236e8a0de1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:34.874063 12222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "d009160d49574d18a5bce9042a90adeb"
format_stamp: "Formatted at 2026-08-11 12:09:34 on dist-test-slave-nj21"
server_key: "24d268e4ec7448ac366d4ec77289b378"
server_key_iv: "5d8170384fcebe491336e236e8a0de1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:34.878881 12222 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 12:09:34.882074 12243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:34.883298 12222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:09:34.883448 12222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "d009160d49574d18a5bce9042a90adeb"
format_stamp: "Formatted at 2026-08-11 12:09:34 on dist-test-slave-nj21"
server_key: "24d268e4ec7448ac366d4ec77289b378"
server_key_iv: "5d8170384fcebe491336e236e8a0de1c"
server_key_version: "encryptionkey@0"
I20260811 12:09:34.883636 12222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:34.888846 12222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:34.889552 12222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:34.889820 12222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:34.906651 12222 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:37559
I20260811 12:09:34.906687 12317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:37559 every 8 connection(s)
I20260811 12:09:34.907943 12222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:34.911590 12318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:34.915489 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 12222
I20260811 12:09:34.915702 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:34.916527 4291 external_mini_cluster.cc:1428] Setting key 0ef842cec65e62861c4764ed58a39952
I20260811 12:09:34.923120 12318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: Bootstrap starting.
I20260811 12:09:34.925110 12322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:34.918546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.4.48.254:37559)
0811 12:09:34.919458 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:34.919468 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:34.919509 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:09:34.921689 (+ 2180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:34.921712 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:34.922091 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:34.922396 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:34.923528 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:34.923534 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:34.923547 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:34.923562 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:34.923633 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:34.923649 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:34.923870 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:34.923972 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 12:09:34.924196 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":347,"threads_started":1}
I20260811 12:09:34.926443 12318 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:34.927466 12318 log.cc:824] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:34.930078 12318 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: No bootstrap required, opened a new log
I20260811 12:09:34.937853 12318 raft_consensus.cc:348] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } }
I20260811 12:09:34.938198 12318 raft_consensus.cc:374] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:34.938300 12318 raft_consensus.cc:729] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d009160d49574d18a5bce9042a90adeb, State: Initialized, Role: FOLLOWER
I20260811 12:09:34.938819 12318 consensus_queue.cc:260] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } }
I20260811 12:09:34.939054 12318 raft_consensus.cc:388] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:34.939183 12318 raft_consensus.cc:482] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:34.939339 12318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:34.941509 12318 raft_consensus.cc:504] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } }
I20260811 12:09:34.941979 12318 leader_election.cc:302] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d009160d49574d18a5bce9042a90adeb; no voters:
I20260811 12:09:34.942533 12318 leader_election.cc:288] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:34.942675 12326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:34.943420 12326 raft_consensus.cc:686] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [term 1 LEADER]: Becoming Leader. State: Replica: d009160d49574d18a5bce9042a90adeb, State: Running, Role: LEADER
I20260811 12:09:34.944012 12326 consensus_queue.cc:237] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } }
I20260811 12:09:34.944486 12318 sys_catalog.cc:564] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:34.946740 12327 sys_catalog.cc:455] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d009160d49574d18a5bce9042a90adeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } } }
I20260811 12:09:34.947144 12327 sys_catalog.cc:458] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:34.947274 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader d009160d49574d18a5bce9042a90adeb. Latest consensus state: current_term: 1 leader_uuid: "d009160d49574d18a5bce9042a90adeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d009160d49574d18a5bce9042a90adeb" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 37559 } } }
I20260811 12:09:34.947651 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:34.953070 12335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:34.957760 12335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:34.966159 12335 catalog_manager.cc:1347] Generated new cluster ID: 2013711f751246f38ac406899613cac2
I20260811 12:09:34.966279 12335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:35.004297 12335 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:35.004735 12335 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:35.014156 12335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: Generated new TSK 0
I20260811 12:09:35.015004 12335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:09:35.026435 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--builtin_ntp_servers=127.4.48.212:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 12:09:35.350813 12361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:35.351050 12361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:35.351176 12361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:35.358282 12361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:35.358366 12361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:35.358501 12361 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:35.367122 12361 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:35.368250 12361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:35.369921 12361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:35.382094 12367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:35.382128 12370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:35.382256 12373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:35.382973 12361 server_base.cc:1034] running on GCE node
I20260811 12:09:35.383502 12361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:35.384409 12361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:35.385619 12361 hybrid_clock.cc:648] HybridClock initialized: now 1786450175385571 us; error 42 us; skew 500 ppm
I20260811 12:09:35.385928 12361 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:35.388264 12361 webserver.cc:460] Webserver started at http://127.4.48.193:42495/ using document root <none> and password file <none>
I20260811 12:09:35.388906 12361 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:35.389004 12361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:35.389302 12361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:35.392067 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "a2a49a489e38400e9119d2ea8799e1f4"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "2b29d4bc37e753e49202838cb35001f0"
server_key_iv: "3be9b2fcb278a0db20e124a42d412a04"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.392804 12361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "a2a49a489e38400e9119d2ea8799e1f4"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "2b29d4bc37e753e49202838cb35001f0"
server_key_iv: "3be9b2fcb278a0db20e124a42d412a04"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.397608 12361 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:35.400787 12381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.401971 12361 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:09:35.402107 12361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "a2a49a489e38400e9119d2ea8799e1f4"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "2b29d4bc37e753e49202838cb35001f0"
server_key_iv: "3be9b2fcb278a0db20e124a42d412a04"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.402316 12361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:35.407603 12361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:35.408347 12361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:35.408587 12361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:35.409425 12361 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:35.410743 12361 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:35.410837 12361 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.410938 12361 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:35.411005 12361 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.440922 12361 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:37805
I20260811 12:09:35.441021 12565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:37805 every 8 connection(s)
I20260811 12:09:35.442214 12361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:35.448165 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 12361
I20260811 12:09:35.448330 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:35.449033 4291 external_mini_cluster.cc:1428] Setting key 0103fe961dcd79ceb828a9a6997a2bda
I20260811 12:09:35.450263 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:35.444679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:35163 (local address 127.4.48.254:37559)
0811 12:09:35.445108 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:35.445112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:35.446243 (+ 1131us) server_negotiation.cc:407] Connection header received
0811 12:09:35.447904 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:35.447910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:35.447977 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:35.448089 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:35.449509 (+ 1420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:35.449515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:35.449520 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:35.449539 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:35.449580 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:35.449591 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:35.449701 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:35.449931 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 12:09:35.449985 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":230,"threads_started":1}
I20260811 12:09:35.451117 12574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:35.445180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:37559 (local address 127.4.48.193:35163)
0811 12:09:35.446092 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:35.446137 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:35.447616 (+ 1479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:35.448334 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:35.448345 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:35.448452 (+ 107us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:35.449205 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:35.449208 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:35.449253 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:35.449335 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:35.449684 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:35.449694 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:35.449702 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:09:35.449969 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 12:09:35.450175 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":280,"threads_started":1}
I20260811 12:09:35.451892 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--builtin_ntp_servers=127.4.48.212:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:09:35.452572 12568 heartbeater.cc:347] Connected to a master server at 127.4.48.254:37559
I20260811 12:09:35.452839 12568 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:35.453433 12568 heartbeater.cc:511] Master 127.4.48.254:37559 requested a full tablet report, sending...
I20260811 12:09:35.455518 12271 ts_manager.cc:194] Registered new tserver with Master: a2a49a489e38400e9119d2ea8799e1f4 (127.4.48.193:37805)
W20260811 12:09:35.772259 12575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:35.772506 12575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:35.772590 12575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:35.779424 12575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:35.779507 12575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:35.779621 12575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:35.788223 12575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:35.789384 12575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:35.791060 12575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:35.803427 12586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:35.803450 12591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:35.803480 12587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:35.804248 12575 server_base.cc:1034] running on GCE node
I20260811 12:09:35.804775 12575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:35.805644 12575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:35.806847 12575 hybrid_clock.cc:648] HybridClock initialized: now 1786450175806772 us; error 58 us; skew 500 ppm
I20260811 12:09:35.807145 12575 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:35.809280 12575 webserver.cc:460] Webserver started at http://127.4.48.194:46303/ using document root <none> and password file <none>
I20260811 12:09:35.809924 12575 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:35.810026 12575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:35.810333 12575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:35.813151 12575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "b21a173cccdb4fcb9594862002dbeeda"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "8482accdc4d1186f14d8b219733d9252"
server_key_iv: "6851c517d1dcae32f406c0d3548f4fac"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.813922 12575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "b21a173cccdb4fcb9594862002dbeeda"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "8482accdc4d1186f14d8b219733d9252"
server_key_iv: "6851c517d1dcae32f406c0d3548f4fac"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.818883 12575 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:09:35.822283 12604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.823550 12575 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:09:35.823727 12575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "b21a173cccdb4fcb9594862002dbeeda"
format_stamp: "Formatted at 2026-08-11 12:09:35 on dist-test-slave-nj21"
server_key: "8482accdc4d1186f14d8b219733d9252"
server_key_iv: "6851c517d1dcae32f406c0d3548f4fac"
server_key_version: "encryptionkey@0"
I20260811 12:09:35.823953 12575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:35.829519 12575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:35.830271 12575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:35.830516 12575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:35.831411 12575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:35.832831 12575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:35.832927 12575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.833034 12575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:35.833106 12575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:35.863726 12575 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:36605
I20260811 12:09:35.863811 12805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:36605 every 8 connection(s)
I20260811 12:09:35.865114 12575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:35.873775 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 12575
I20260811 12:09:35.873914 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:35.868502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:45653 (local address 127.4.48.254:37559)
0811 12:09:35.868696 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:35.868701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:35.870128 (+ 1427us) server_negotiation.cc:407] Connection header received
0811 12:09:35.871808 (+ 1680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:35.871813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:35.871897 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:35.871991 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:35.873177 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:35.873183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:35.873187 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:35.873203 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:35.873235 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:35.873246 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:35.873359 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:35.873550 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 12:09:35.873617 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:09:35.873934 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:35.874415 4291 external_mini_cluster.cc:1428] Setting key aea886e7eefb32453ef298335917b878
I20260811 12:09:35.874805 12814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:35.868996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:37559 (local address 127.4.48.194:45653)
0811 12:09:35.869959 (+ 963us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:35.870003 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:35.871542 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:35.872217 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:35.872225 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:35.872309 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:35.872897 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:35.872900 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:35.872935 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:35.873014 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:35.873332 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:35.873344 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:35.873352 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:09:35.873621 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 12:09:35.873876 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":316,"threads_started":1}
I20260811 12:09:35.876206 12806 heartbeater.cc:347] Connected to a master server at 127.4.48.254:37559
I20260811 12:09:35.876485 12806 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:35.877031 12806 heartbeater.cc:511] Master 127.4.48.254:37559 requested a full tablet report, sending...
I20260811 12:09:35.877183 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--builtin_ntp_servers=127.4.48.212:38805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 12:09:35.878392 12271 ts_manager.cc:194] Registered new tserver with Master: b21a173cccdb4fcb9594862002dbeeda (127.4.48.194:36605)
W20260811 12:09:36.195055 12815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:36.195314 12815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:36.195405 12815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:36.202656 12815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:36.202746 12815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:36.202857 12815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:36.211416 12815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:38805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:37559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:36.212549 12815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:36.214205 12815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:36.226167 12820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:36.226212 12825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:36.226238 12821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:36.226986 12815 server_base.cc:1034] running on GCE node
I20260811 12:09:36.227538 12815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:36.228519 12815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:36.229713 12815 hybrid_clock.cc:648] HybridClock initialized: now 1786450176229661 us; error 45 us; skew 500 ppm
I20260811 12:09:36.230021 12815 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:36.232184 12815 webserver.cc:460] Webserver started at http://127.4.48.195:39729/ using document root <none> and password file <none>
I20260811 12:09:36.232820 12815 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:36.232918 12815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:36.233209 12815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:36.235910 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "1de532f977f947828c38e3289f14f69e"
format_stamp: "Formatted at 2026-08-11 12:09:36 on dist-test-slave-nj21"
server_key: "f1869938221b97ec93e00f367dee7798"
server_key_iv: "2ba9cf0e453f55d5cafb940b86b7b6c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:36.236637 12815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "1de532f977f947828c38e3289f14f69e"
format_stamp: "Formatted at 2026-08-11 12:09:36 on dist-test-slave-nj21"
server_key: "f1869938221b97ec93e00f367dee7798"
server_key_iv: "2ba9cf0e453f55d5cafb940b86b7b6c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:36.241554 12815 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:09:36.244855 12836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:36.246094 12815 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:36.246243 12815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "1de532f977f947828c38e3289f14f69e"
format_stamp: "Formatted at 2026-08-11 12:09:36 on dist-test-slave-nj21"
server_key: "f1869938221b97ec93e00f367dee7798"
server_key_iv: "2ba9cf0e453f55d5cafb940b86b7b6c5"
server_key_version: "encryptionkey@0"
I20260811 12:09:36.246448 12815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:36.251982 12815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:36.252701 12815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:36.253074 12815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:36.253978 12815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:36.255503 12815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:36.255600 12815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:36.255734 12815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:36.255823 12815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:36.285147 12815 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:41107
I20260811 12:09:36.285194 13021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:41107 every 8 connection(s)
I20260811 12:09:36.286427 12815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:36.288686 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 12815
I20260811 12:09:36.288858 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:36.289426 4291 external_mini_cluster.cc:1428] Setting key dbacb3120831bdc6b9ca251c57c45db2
I20260811 12:09:36.295279 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.289067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:38051 (local address 127.4.48.254:37559)
0811 12:09:36.289235 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.289239 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.291053 (+ 1814us) server_negotiation.cc:407] Connection header received
0811 12:09:36.293166 (+ 2113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.293172 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.293237 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.293322 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.294576 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.294581 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.294585 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.294600 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.294636 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.294648 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.294764 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.294937 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.295004 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:09:36.296229 13025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.289785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:37559 (local address 127.4.48.195:38051)
0811 12:09:36.290900 (+ 1115us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.290954 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.292922 (+ 1968us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.293557 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.293565 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.293660 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.294261 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.294264 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.294299 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.294415 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.294733 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.294744 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.294752 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.295015 (+ 263us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.295272 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":339,"threads_started":1}
I20260811 12:09:36.297324 13022 heartbeater.cc:347] Connected to a master server at 127.4.48.254:37559
I20260811 12:09:36.297569 13022 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:36.298074 13022 heartbeater.cc:511] Master 127.4.48.254:37559 requested a full tablet report, sending...
I20260811 12:09:36.299351 12271 ts_manager.cc:194] Registered new tserver with Master: 1de532f977f947828c38e3289f14f69e (127.4.48.195:41107)
I20260811 12:09:36.304687 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:36.311108 12573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.308986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57240 (local address 127.4.48.254:37559)
0811 12:09:36.309173 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.309180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.309593 (+ 413us) server_negotiation.cc:407] Connection header received
0811 12:09:36.309730 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.309733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.309784 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.309852 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.310609 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.310613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.310616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.310631 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.310660 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.310670 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.310760 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.310791 (+ 31us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.310833 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:09:36.314826 12271 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:36.321425 12271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:36.337888 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.332933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51214 (local address 127.4.48.194:36605)
0811 12:09:36.334129 (+ 1196us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.334139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.334169 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:09:36.334240 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.334248 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.334494 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.335260 (+ 766us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.336828 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.336838 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.336852 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.336867 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.336923 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.336947 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.337124 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.337450 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.337562 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1015,"thread_start_us":884,"threads_started":1}
I20260811 12:09:36.337955 13037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.332698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36605 (local address 127.0.0.1:51214)
0811 12:09:36.333452 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.333488 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.333623 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.334714 (+ 1091us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.334721 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.334817 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.335573 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.335581 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.335617 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.335736 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.337069 (+ 1333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.337080 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.337088 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.337275 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.337427 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":365,"threads_started":1}
I20260811 12:09:36.337991 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.334145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44432 (local address 127.4.48.195:41107)
0811 12:09:36.335038 (+ 893us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.335044 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.335069 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:09:36.335119 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.335124 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.335297 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.335953 (+ 656us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.336993 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.336999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.337011 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.337026 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.337074 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.337093 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.337481 (+ 388us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.337540 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.337613 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":605,"threads_started":1}
I20260811 12:09:36.338364 13039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.333803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41107 (local address 127.0.0.1:44432)
0811 12:09:36.334391 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.334414 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.334522 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.335477 (+ 955us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.335487 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.335512 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.336166 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.336169 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.336181 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.336216 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.337211 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.337215 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.337220 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.337422 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.337475 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":325,"threads_started":1}
I20260811 12:09:36.341253 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.334986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.4.48.193:37805)
0811 12:09:36.335863 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.336109 (+ 246us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.338187 (+ 2078us) server_negotiation.cc:407] Connection header received
0811 12:09:36.338557 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.338566 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.338743 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.338887 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.340472 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.340481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.340494 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.340522 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.340582 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.340604 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.340745 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.340873 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.340963 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":558,"threads_started":1}
I20260811 12:09:36.341471 13041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.334472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:37805 (local address 127.0.0.1:56494)
0811 12:09:36.337785 (+ 3313us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.337805 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.337932 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.339536 (+ 1604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.339539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.339557 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.340237 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.340240 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.340257 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.340311 (+ 54us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.340719 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.340724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.340728 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.340857 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.340905 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3203,"thread_start_us":2828,"threads_started":1}
I20260811 12:09:36.342445 12916 tablet_service.cc:1467] Processing CreateTablet for tablet 04345605572e4bb8980abe2a1f932094 (DEFAULT_TABLE table=test-table [id=16a37fd6dd204be7a57c4eeac9712359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:36.342347 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 04345605572e4bb8980abe2a1f932094 (DEFAULT_TABLE table=test-table [id=16a37fd6dd204be7a57c4eeac9712359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:36.343636 12916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04345605572e4bb8980abe2a1f932094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:36.343781 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04345605572e4bb8980abe2a1f932094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:36.344787 12462 tablet_service.cc:1467] Processing CreateTablet for tablet 04345605572e4bb8980abe2a1f932094 (DEFAULT_TABLE table=test-table [id=16a37fd6dd204be7a57c4eeac9712359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:36.346065 12462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 04345605572e4bb8980abe2a1f932094. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:36.355808 13044 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Bootstrap starting.
I20260811 12:09:36.358146 13045 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Bootstrap starting.
I20260811 12:09:36.359596 13044 tablet_bootstrap.cc:654] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:36.359731 13047 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Bootstrap starting.
I20260811 12:09:36.361334 13044 log.cc:824] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:36.362035 13045 tablet_bootstrap.cc:654] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:36.363093 13047 tablet_bootstrap.cc:654] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:36.363047 13045 log.cc:824] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:36.364204 13047 log.cc:824] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:36.364329 13044 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: No bootstrap required, opened a new log
I20260811 12:09:36.364598 13044 ts_tablet_manager.cc:1397] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:09:36.366146 13045 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: No bootstrap required, opened a new log
I20260811 12:09:36.366459 13045 ts_tablet_manager.cc:1397] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:09:36.366789 13047 tablet_bootstrap.cc:492] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: No bootstrap required, opened a new log
I20260811 12:09:36.367005 13047 ts_tablet_manager.cc:1397] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:09:36.373921 13047 raft_consensus.cc:348] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.374254 13047 raft_consensus.cc:374] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:36.374344 13047 raft_consensus.cc:729] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2a49a489e38400e9119d2ea8799e1f4, State: Initialized, Role: FOLLOWER
I20260811 12:09:36.374511 13044 raft_consensus.cc:348] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.374907 13044 raft_consensus.cc:374] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:36.375008 13044 raft_consensus.cc:729] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1de532f977f947828c38e3289f14f69e, State: Initialized, Role: FOLLOWER
I20260811 12:09:36.374909 13047 consensus_queue.cc:260] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.375698 13044 consensus_queue.cc:260] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.376014 12568 heartbeater.cc:503] Master 127.4.48.254:37559 was elected leader, sending a full tablet report...
I20260811 12:09:36.376654 13047 ts_tablet_manager.cc:1428] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 12:09:36.376740 13022 heartbeater.cc:503] Master 127.4.48.254:37559 was elected leader, sending a full tablet report...
I20260811 12:09:36.377351 13044 ts_tablet_manager.cc:1428] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:09:36.378028 13045 raft_consensus.cc:348] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.378455 13045 raft_consensus.cc:374] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:36.378558 13045 raft_consensus.cc:729] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b21a173cccdb4fcb9594862002dbeeda, State: Initialized, Role: FOLLOWER
I20260811 12:09:36.379046 13045 consensus_queue.cc:260] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.380039 12806 heartbeater.cc:503] Master 127.4.48.254:37559 was elected leader, sending a full tablet report...
I20260811 12:09:36.380676 13045 ts_tablet_manager.cc:1428] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 12:09:36.384117 13052 raft_consensus.cc:482] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:36.384382 13052 raft_consensus.cc:504] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.385727 13052 leader_election.cc:288] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b21a173cccdb4fcb9594862002dbeeda (127.4.48.194:36605), 1de532f977f947828c38e3289f14f69e (127.4.48.195:41107)
I20260811 12:09:36.388788 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.386016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:58821 (local address 127.4.48.194:36605)
0811 12:09:36.386163 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.386167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.386772 (+ 605us) server_negotiation.cc:407] Connection header received
0811 12:09:36.386944 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.386951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.387016 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.387124 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.388250 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.388253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.388256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.388271 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.388300 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.388311 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.388419 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.388521 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.388580 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 12:09:36.388999 13055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.386061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36605 (local address 127.4.48.193:58821)
0811 12:09:36.386659 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.386677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.386797 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.387220 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.387229 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.387253 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.388046 (+ 793us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.388049 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.388065 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.388105 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.388410 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.388416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.388421 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.388538 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.388589 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":299,"threads_started":1}
I20260811 12:09:36.389709 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.386968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56647 (local address 127.4.48.195:41107)
0811 12:09:36.387124 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.387129 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.387567 (+ 438us) server_negotiation.cc:407] Connection header received
0811 12:09:36.387788 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.387793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.387840 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.387922 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.389030 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.389036 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.389040 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.389059 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.389093 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.389109 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.389203 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.389412 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.389479 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:09:36.389704 12719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04345605572e4bb8980abe2a1f932094" candidate_uuid: "a2a49a489e38400e9119d2ea8799e1f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21a173cccdb4fcb9594862002dbeeda" is_pre_election: true
I20260811 12:09:36.389895 13058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.386735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41107 (local address 127.4.48.193:56647)
0811 12:09:36.387476 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:36.387493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:36.387615 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:36.388073 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:36.388077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:36.388097 (+ 20us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:36.388725 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:09:36.388728 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:36.388744 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:09:36.388784 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:36.389287 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:36.389292 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:36.389296 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 12:09:36.389467 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 12:09:36.389513 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":409,"threads_started":1}
I20260811 12:09:36.390094 12719 raft_consensus.cc:2454] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a49a489e38400e9119d2ea8799e1f4 in term 0.
I20260811 12:09:36.390640 12384 leader_election.cc:302] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2a49a489e38400e9119d2ea8799e1f4, b21a173cccdb4fcb9594862002dbeeda; no voters:
I20260811 12:09:36.390657 12952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04345605572e4bb8980abe2a1f932094" candidate_uuid: "a2a49a489e38400e9119d2ea8799e1f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de532f977f947828c38e3289f14f69e" is_pre_election: true
I20260811 12:09:36.390972 13052 raft_consensus.cc:2781] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:36.391088 13052 raft_consensus.cc:482] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:36.391054 12952 raft_consensus.cc:2454] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2a49a489e38400e9119d2ea8799e1f4 in term 0.
I20260811 12:09:36.391188 13052 raft_consensus.cc:3037] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:36.393692 13052 raft_consensus.cc:504] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.394245 13052 leader_election.cc:288] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [CANDIDATE]: Term 1 election: Requested vote from peers b21a173cccdb4fcb9594862002dbeeda (127.4.48.194:36605), 1de532f977f947828c38e3289f14f69e (127.4.48.195:41107)
I20260811 12:09:36.394589 12719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04345605572e4bb8980abe2a1f932094" candidate_uuid: "a2a49a489e38400e9119d2ea8799e1f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b21a173cccdb4fcb9594862002dbeeda"
I20260811 12:09:36.394712 12952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "04345605572e4bb8980abe2a1f932094" candidate_uuid: "a2a49a489e38400e9119d2ea8799e1f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1de532f977f947828c38e3289f14f69e"
I20260811 12:09:36.394786 12719 raft_consensus.cc:3037] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:36.394892 12952 raft_consensus.cc:3037] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:36.397971 12719 raft_consensus.cc:2454] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a49a489e38400e9119d2ea8799e1f4 in term 1.
I20260811 12:09:36.397964 12952 raft_consensus.cc:2454] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2a49a489e38400e9119d2ea8799e1f4 in term 1.
I20260811 12:09:36.398309 12384 leader_election.cc:302] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1de532f977f947828c38e3289f14f69e, a2a49a489e38400e9119d2ea8799e1f4; no voters:
I20260811 12:09:36.398562 13052 raft_consensus.cc:2781] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:36.399004 13052 raft_consensus.cc:686] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 1 LEADER]: Becoming Leader. State: Replica: a2a49a489e38400e9119d2ea8799e1f4, State: Running, Role: LEADER
I20260811 12:09:36.399451 13052 consensus_queue.cc:237] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } }
I20260811 12:09:36.403466 12268 catalog_manager.cc:5495] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2a49a489e38400e9119d2ea8799e1f4 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "a2a49a489e38400e9119d2ea8799e1f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2a49a489e38400e9119d2ea8799e1f4" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 37805 } health_report { overall_health: HEALTHY } } }
I20260811 12:09:36.428946 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.427054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56502 (local address 127.4.48.193:37805)
0811 12:09:36.427196 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.427200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.427348 (+ 148us) server_negotiation.cc:407] Connection header received
0811 12:09:36.427549 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.427553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.427625 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.427748 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.428456 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.428460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.428463 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.428477 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.428508 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.428519 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.428601 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.428687 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.428730 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:09:36.436444 12719 raft_consensus.cc:1261] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 1 FOLLOWER]: Refusing update from remote peer a2a49a489e38400e9119d2ea8799e1f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:36.436446 12952 raft_consensus.cc:1261] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 1 FOLLOWER]: Refusing update from remote peer a2a49a489e38400e9119d2ea8799e1f4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:36.437294 13060 consensus_queue.cc:1035] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b21a173cccdb4fcb9594862002dbeeda" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:36.437614 13052 consensus_queue.cc:1035] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1de532f977f947828c38e3289f14f69e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:09:36.444275 12569 tablet.cc:2369] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:36.468629 12268 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:36.468924 12268 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:36.472702 12268 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d009160d49574d18a5bce9042a90adeb: Sending DeleteTablet for 3 replicas of tablet 04345605572e4bb8980abe2a1f932094
I20260811 12:09:36.473685 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 04345605572e4bb8980abe2a1f932094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:36 UTC) from {username='slave'} at 127.0.0.1:51214
I20260811 12:09:36.473973 12916 tablet_service.cc:1514] Processing DeleteTablet for tablet 04345605572e4bb8980abe2a1f932094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:36 UTC) from {username='slave'} at 127.0.0.1:44432
I20260811 12:09:36.473989 12462 tablet_service.cc:1514] Processing DeleteTablet for tablet 04345605572e4bb8980abe2a1f932094 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:36 UTC) from {username='slave'} at 127.0.0.1:56494
I20260811 12:09:36.476087 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 12361
I20260811 12:09:36.486654 13084 tablet_replica.cc:330] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: stopping tablet replica
I20260811 12:09:36.487164 13084 raft_consensus.cc:2229] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:36.487283 13091 tablet_replica.cc:330] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: stopping tablet replica
I20260811 12:09:36.487599 13091 raft_consensus.cc:2229] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:36.487727 13084 raft_consensus.cc:2258] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:36.488243 13091 raft_consensus.cc:2258] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:36.489880 13084 ts_tablet_manager.cc:1905] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:36.490314 13091 ts_tablet_manager.cc:1905] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:36.497064 13084 ts_tablet_manager.cc:1918] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:36.497323 13084 log.cc:1196] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/04345605572e4bb8980abe2a1f932094
I20260811 12:09:36.497576 13091 ts_tablet_manager.cc:1918] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:36.497737 13091 log.cc:1196] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/04345605572e4bb8980abe2a1f932094
I20260811 12:09:36.498061 13084 ts_tablet_manager.cc:1939] T 04345605572e4bb8980abe2a1f932094 P b21a173cccdb4fcb9594862002dbeeda: Deleting consensus metadata
I20260811 12:09:36.498284 13091 ts_tablet_manager.cc:1939] T 04345605572e4bb8980abe2a1f932094 P a2a49a489e38400e9119d2ea8799e1f4: Deleting consensus metadata
I20260811 12:09:36.500006 12245 catalog_manager.cc:4841] TS b21a173cccdb4fcb9594862002dbeeda (127.4.48.194:36605): tablet 04345605572e4bb8980abe2a1f932094 (table test-table [id=16a37fd6dd204be7a57c4eeac9712359]) successfully deleted
I20260811 12:09:36.500332 12245 catalog_manager.cc:4841] TS a2a49a489e38400e9119d2ea8799e1f4 (127.4.48.193:37805): tablet 04345605572e4bb8980abe2a1f932094 (table test-table [id=16a37fd6dd204be7a57c4eeac9712359]) successfully deleted
I20260811 12:09:36.502148 13089 tablet_replica.cc:330] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: stopping tablet replica
I20260811 12:09:36.502568 13089 raft_consensus.cc:2229] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:36.503113 13089 raft_consensus.cc:2258] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:36.503634 13043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.477292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.4.48.193:37805)
0811 12:09:36.477461 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.477467 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.501444 (+ 23977us) server_negotiation.cc:407] Connection header received
0811 12:09:36.501606 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.501612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.501675 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.501785 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.502837 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.502843 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.502847 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.502867 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.502904 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.502919 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.503013 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.503282 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.503360 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:36.504814 12558 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:36.505084 13089 ts_tablet_manager.cc:1905] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:36.510826 13089 ts_tablet_manager.cc:1918] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:36.510954 13089 log.cc:1196] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/04345605572e4bb8980abe2a1f932094
I20260811 12:09:36.511385 13089 ts_tablet_manager.cc:1939] T 04345605572e4bb8980abe2a1f932094 P 1de532f977f947828c38e3289f14f69e: Deleting consensus metadata
I20260811 12:09:36.512759 12245 catalog_manager.cc:4841] TS 1de532f977f947828c38e3289f14f69e (127.4.48.195:41107): tablet 04345605572e4bb8980abe2a1f932094 (table test-table [id=16a37fd6dd204be7a57c4eeac9712359]) successfully deleted
I20260811 12:09:36.598027 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 12361
I20260811 12:09:36.625689 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 12575
I20260811 12:09:36.628360 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.626500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51228 (local address 127.4.48.194:36605)
0811 12:09:36.626659 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.626664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.626682 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:09:36.626816 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.626822 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.626917 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.626996 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.627824 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.627829 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.627832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.627846 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.627882 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.627897 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.628008 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.628049 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.628092 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:09:36.628995 12797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:36.726424 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 12575
I20260811 12:09:36.748066 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 12815
I20260811 12:09:36.750679 13040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:36.748819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44438 (local address 127.4.48.195:41107)
0811 12:09:36.748979 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:36.748983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:36.749002 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:09:36.749149 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:36.749153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:36.749238 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:36.749319 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:36.750178 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:36.750183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:36.750186 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:36.750201 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:36.750240 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:36.750252 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:36.750364 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:36.750407 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 12:09:36.750453 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:09:36.751304 13015 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:36.836052 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 12815
I20260811 12:09:36.860296 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 12222
I20260811 12:09:36.861410 12312 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:36.964998 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 12222
2026-08-11T12:09:36Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2528 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 12:09:37.010545 4291 mini_oidc.cc:152] Starting JWKS server
I20260811 12:09:37.013469 4291 webserver.cc:460] Webserver started at https://127.0.0.1:35121/ using document root <none> and password file <none>
I20260811 12:09:37.013814 4291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:09:37.014428 4291 webserver.cc:460] Webserver started at http://127.0.0.1:40229/ using document root <none> and password file <none>
2026-08-11T12:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:37Z Disabled control of system clock
I20260811 12:09:37.043996 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:32769
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:32769
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35121/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/testchainca.cert with env {}
W20260811 12:09:37.359949 13114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:37.360195 13114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:37.360283 13114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:37.367049 13114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:37.367136 13114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:37.367184 13114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:37.367237 13114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:37.367278 13114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:09:37.367337 13114 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:09:37.367376 13114 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35121/jwks.json
I20260811 12:09:37.376546 13114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:32769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:32769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35121/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:37.377590 13114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:37.378731 13114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:37.389526 13120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:37.389604 13125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:37.389577 13122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:37.390228 13114 server_base.cc:1034] running on GCE node
I20260811 12:09:37.390914 13114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:37.392181 13114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:37.393380 13114 hybrid_clock.cc:648] HybridClock initialized: now 1786450177393340 us; error 30 us; skew 500 ppm
I20260811 12:09:37.393703 13114 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:37.399889 13114 webserver.cc:460] Webserver started at http://127.4.48.254:44453/ using document root <none> and password file <none>
I20260811 12:09:37.400578 13114 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:37.400677 13114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:37.400971 13114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:37.403764 13114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "e69fe85aba2d49f592d1981a022044a2"
format_stamp: "Formatted at 2026-08-11 12:09:37 on dist-test-slave-nj21"
server_key: "bb021228d3a9ca453a96dc31e98cd418"
server_key_iv: "033d171c421490f64cc14c98c308443d"
server_key_version: "encryptionkey@0"
I20260811 12:09:37.404598 13114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "e69fe85aba2d49f592d1981a022044a2"
format_stamp: "Formatted at 2026-08-11 12:09:37 on dist-test-slave-nj21"
server_key: "bb021228d3a9ca453a96dc31e98cd418"
server_key_iv: "033d171c421490f64cc14c98c308443d"
server_key_version: "encryptionkey@0"
I20260811 12:09:37.409231 13114 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:09:37.412429 13134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:37.413563 13114 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:09:37.413718 13114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "e69fe85aba2d49f592d1981a022044a2"
format_stamp: "Formatted at 2026-08-11 12:09:37 on dist-test-slave-nj21"
server_key: "bb021228d3a9ca453a96dc31e98cd418"
server_key_iv: "033d171c421490f64cc14c98c308443d"
server_key_version: "encryptionkey@0"
I20260811 12:09:37.413916 13114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:37.430310 13114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:37.431053 13114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:37.431336 13114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:37.448009 13114 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:32769
I20260811 12:09:37.448031 13213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:32769 every 8 connection(s)
I20260811 12:09:37.449203 13114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:37.451211 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13114
I20260811 12:09:37.451395 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:37.452080 4291 external_mini_cluster.cc:1428] Setting key 91283802f983e06f10bcf61bc3a6fe32
I20260811 12:09:37.453011 13214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:37.466554 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:37.453821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.4.48.254:32769)
0811 12:09:37.454878 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:37.454892 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:37.454943 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:09:37.456614 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:37.456642 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:37.457218 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:37.457642 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:37.458143 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.460155 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.461447 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.462397 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.462561 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:37.463594 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:37.463604 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:37.463625 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:37.464366 (+ 741us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:37.464460 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:37.464486 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:37.464603 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:37.464803 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:09:37.465133 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":311,"threads_started":1}
I20260811 12:09:37.467697 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2: Bootstrap starting.
I20260811 12:09:37.470053 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:09:37.471035 13214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:37.472205 13214 log.cc:824] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:37.474949 13214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2: No bootstrap required, opened a new log
I20260811 12:09:37.482117 13214 raft_consensus.cc:348] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } }
I20260811 12:09:37.482443 13214 raft_consensus.cc:374] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:37.482519 13214 raft_consensus.cc:729] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69fe85aba2d49f592d1981a022044a2, State: Initialized, Role: FOLLOWER
I20260811 12:09:37.483000 13214 consensus_queue.cc:260] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } }
I20260811 12:09:37.483208 13214 raft_consensus.cc:388] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:37.483301 13214 raft_consensus.cc:482] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:37.483420 13214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:37.485581 13214 raft_consensus.cc:504] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } }
I20260811 12:09:37.486032 13214 leader_election.cc:302] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e69fe85aba2d49f592d1981a022044a2; no voters:
I20260811 12:09:37.486521 13214 leader_election.cc:288] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:37.486658 13220 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:37.487294 13220 raft_consensus.cc:686] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [term 1 LEADER]: Becoming Leader. State: Replica: e69fe85aba2d49f592d1981a022044a2, State: Running, Role: LEADER
I20260811 12:09:37.487823 13220 consensus_queue.cc:237] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } }
I20260811 12:09:37.488528 13214 sys_catalog.cc:564] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:37.490778 13221 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e69fe85aba2d49f592d1981a022044a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } } }
I20260811 12:09:37.491185 13221 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:37.491212 13222 sys_catalog.cc:455] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e69fe85aba2d49f592d1981a022044a2. Latest consensus state: current_term: 1 leader_uuid: "e69fe85aba2d49f592d1981a022044a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e69fe85aba2d49f592d1981a022044a2" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32769 } } }
I20260811 12:09:37.492336 13222 sys_catalog.cc:458] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:37.495025 13230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:37.500242 13230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:37.508934 13230 catalog_manager.cc:1347] Generated new cluster ID: b2b632f4e47e474fa6ec543c23d86a95
I20260811 12:09:37.509038 13230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:37.521391 13230 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:37.522508 13230 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:37.529034 13230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e69fe85aba2d49f592d1981a022044a2: Generated new TSK 0
I20260811 12:09:37.529718 13230 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:09:37.576016 13258 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 12:09:37.576237 13258 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:09:37.578298 13217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:37.548999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53680 (local address 127.4.48.254:32769)
0811 12:09:37.549136 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:37.549141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:37.549760 (+ 619us) server_negotiation.cc:407] Connection header received
0811 12:09:37.550114 (+ 354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:37.550119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:37.550190 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:37.550322 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:09:37.550867 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.551557 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.552981 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.553437 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.553555 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:37.554062 (+ 507us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:09:37.577607 (+ 23545us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:09:37.577965 (+ 358us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 12:09:37.578431 13217 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53680: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 12:09:37.580157 13257 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:37.549059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:32769 (local address 127.0.0.1:53680)
0811 12:09:37.549651 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:37.549666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:37.549835 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:37.550371 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:37.550376 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:37.550388 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:09:37.550720 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:37.550729 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.551719 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.551725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:37.552836 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:37.552848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:37.553622 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:37.553630 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:37.553764 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:37.553824 (+ 60us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:09:37.577896 (+ 24072us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 12:09:37.578237 (+ 341us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.48.254:32769: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":299,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.4.48.254:32769: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35121/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 12:09:37.581629 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13114
I20260811 12:09:37.582789 13206 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:37.673434 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13114
2026-08-11T12:09:37Z chronyd exiting
I20260811 12:09:37.732005 4291 test_util.cc:175] -----------------------------------------------
I20260811 12:09:37.732156 4291 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786450132843697-4291-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (726 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 12:09:37.736001 4291 mini_oidc.cc:152] Starting JWKS server
I20260811 12:09:37.737879 4291 webserver.cc:460] Webserver started at https://127.0.0.1:43675/ using document root <none> and password file <none>
I20260811 12:09:37.738085 4291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:09:37.738762 4291 webserver.cc:460] Webserver started at http://127.0.0.1:34793/ using document root <none> and password file <none>
2026-08-11T12:09:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:37Z Disabled control of system clock
I20260811 12:09:37.769587 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45223
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:38009
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45223
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43675/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/testchainca.cert with env {}
W20260811 12:09:38.088469 13271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:38.088764 13271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:38.088900 13271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:38.095969 13271 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:38.096072 13271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:38.096138 13271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:38.096218 13271 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:38.096276 13271 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:09:38.096361 13271 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:09:38.096413 13271 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43675/jwks.json
I20260811 12:09:38.106348 13271 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:38009
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45223
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43675/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:38.107561 13271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:38.108841 13271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:38.119894 13280 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:38.119936 13283 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:38.119902 13281 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:38.120668 13271 server_base.cc:1034] running on GCE node
I20260811 12:09:38.121381 13271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:38.122653 13271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:38.123857 13271 hybrid_clock.cc:648] HybridClock initialized: now 1786450178123809 us; error 41 us; skew 500 ppm
I20260811 12:09:38.124204 13271 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:38.126367 13271 webserver.cc:460] Webserver started at http://127.4.48.254:44589/ using document root <none> and password file <none>
I20260811 12:09:38.127013 13271 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:38.127120 13271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:38.127424 13271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:38.130200 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "89c56e46e6774b10ba5b7717e635f930"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "601e764b97670eee4702805be4ff52fb"
server_key_iv: "c385f6aa06705aa3de10acc49c4788bf"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.130959 13271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "89c56e46e6774b10ba5b7717e635f930"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "601e764b97670eee4702805be4ff52fb"
server_key_iv: "c385f6aa06705aa3de10acc49c4788bf"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.135838 13271 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 12:09:38.139201 13290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:38.140578 13271 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:38.140760 13271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "89c56e46e6774b10ba5b7717e635f930"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "601e764b97670eee4702805be4ff52fb"
server_key_iv: "c385f6aa06705aa3de10acc49c4788bf"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.140970 13271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:38.158306 13271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:38.159070 13271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:38.159354 13271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:38.176433 13271 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45223
I20260811 12:09:38.176455 13373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45223 every 8 connection(s)
I20260811 12:09:38.177678 13271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:38.181274 13374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:38.186810 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13271
I20260811 12:09:38.187000 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:38.187644 4291 external_mini_cluster.cc:1428] Setting key 4a345c61bd4d24c46d28aa71ced578d1
I20260811 12:09:38.191629 13374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930: Bootstrap starting.
I20260811 12:09:38.194844 13374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:38.196131 13374 log.cc:824] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:38.199293 13374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930: No bootstrap required, opened a new log
I20260811 12:09:38.201190 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:38.189558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.4.48.254:45223)
0811 12:09:38.190478 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:38.190488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:38.190528 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:09:38.192222 (+ 1694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:38.192244 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:38.192697 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:38.193007 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:38.193622 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.194885 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.196734 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.197684 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.197866 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:38.198988 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:38.198995 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:38.199009 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:38.199544 (+ 535us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:38.199621 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:38.199645 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:38.199771 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:38.199980 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:09:38.200274 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":329,"threads_started":1}
I20260811 12:09:38.204741 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:09:38.206964 13374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } }
I20260811 12:09:38.207285 13374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:38.207399 13374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89c56e46e6774b10ba5b7717e635f930, State: Initialized, Role: FOLLOWER
I20260811 12:09:38.208002 13374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } }
I20260811 12:09:38.208226 13374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:38.208371 13374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:38.208549 13374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:38.210704 13374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } }
I20260811 12:09:38.211155 13374 leader_election.cc:302] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89c56e46e6774b10ba5b7717e635f930; no voters:
I20260811 12:09:38.211695 13374 leader_election.cc:288] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:38.211825 13381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:38.212524 13381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [term 1 LEADER]: Becoming Leader. State: Replica: 89c56e46e6774b10ba5b7717e635f930, State: Running, Role: LEADER
I20260811 12:09:38.213013 13381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } }
I20260811 12:09:38.213758 13374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:38.215634 13383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89c56e46e6774b10ba5b7717e635f930. Latest consensus state: current_term: 1 leader_uuid: "89c56e46e6774b10ba5b7717e635f930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } } }
I20260811 12:09:38.215624 13382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89c56e46e6774b10ba5b7717e635f930" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89c56e46e6774b10ba5b7717e635f930" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45223 } } }
I20260811 12:09:38.216089 13382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:38.216089 13383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:38.217569 13389 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:38.222921 13389 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:38.231482 13389 catalog_manager.cc:1347] Generated new cluster ID: 464b2b76637d4a1e9926e941c5890ae0
I20260811 12:09:38.231588 13389 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:38.265957 13389 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:38.267288 13389 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:38.273569 13389 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89c56e46e6774b10ba5b7717e635f930: Generated new TSK 0
I20260811 12:09:38.274305 13389 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 12:09:38.310232 13421 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 12:09:38.310359 13421 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:09:38.311383 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:38.292995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.4.48.254:45223)
0811 12:09:38.293170 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:38.293177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:38.293572 (+ 395us) server_negotiation.cc:407] Connection header received
0811 12:09:38.293786 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:38.293790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:38.293868 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:38.294001 (+ 133us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:09:38.294471 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.295185 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.296597 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.297052 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.297157 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:38.297623 (+ 466us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:09:38.310787 (+ 13164us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:09:38.311125 (+ 338us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49814: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 12:09:38.311508 13379 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49814: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 12:09:38.311563 13420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:38.292903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45223 (local address 127.0.0.1:49814)
0811 12:09:38.293467 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:38.293482 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:38.293620 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:38.294027 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:38.294031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:38.294045 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:09:38.294300 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:38.294310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.295340 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.295344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:38.296458 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:38.296471 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.297208 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.297215 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:38.297349 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:38.297383 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:09:38.310978 (+ 13595us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 12:09:38.311189 (+ 211us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.48.254:45223: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43675/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":299,"threads_started":1}
I20260811 12:09:38.312253 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13271
I20260811 12:09:38.313572 13369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:38.406602 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13271
2026-08-11T12:09:38Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (732 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 12:09:38.467952 4291 mini_oidc.cc:152] Starting JWKS server
I20260811 12:09:38.469800 4291 webserver.cc:460] Webserver started at https://127.0.0.1:36509/ using document root <none> and password file <none>
I20260811 12:09:38.470052 4291 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 12:09:38.470628 4291 webserver.cc:460] Webserver started at http://127.0.0.1:46119/ using document root <none> and password file <none>
2026-08-11T12:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:38Z Disabled control of system clock
I20260811 12:09:38.502349 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42739
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:37297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:42739
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36509/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:09:38.818461 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:38.818768 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:38.818903 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:38.826151 13438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:38.826288 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:38.826356 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:38.826437 13438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:38.826498 13438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:09:38.826586 13438 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:09:38.826638 13438 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36509/jwks.json
I20260811 12:09:38.835631 13438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:42739
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36509/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:38.837018 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:38.838841 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:38.850158 13449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:38.850205 13444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:38.850232 13446 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:38.851011 13438 server_base.cc:1034] running on GCE node
I20260811 12:09:38.851791 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:38.853106 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:38.854300 13438 hybrid_clock.cc:648] HybridClock initialized: now 1786450178854257 us; error 39 us; skew 500 ppm
I20260811 12:09:38.854619 13438 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:38.856775 13438 webserver.cc:460] Webserver started at http://127.4.48.254:46057/ using document root <none> and password file <none>
I20260811 12:09:38.857393 13438 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:38.857479 13438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:38.857751 13438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:38.860551 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "1673bbad67bd44eeabc84f3efad37bb9"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "270095fed818a93119ebf68012fb92ec"
server_key_iv: "b34ea5f7fb30104e37db2f8649d0dec8"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.861274 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "1673bbad67bd44eeabc84f3efad37bb9"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "270095fed818a93119ebf68012fb92ec"
server_key_iv: "b34ea5f7fb30104e37db2f8649d0dec8"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.866237 13438 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:09:38.869571 13457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:38.870910 13438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:09:38.871055 13438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "1673bbad67bd44eeabc84f3efad37bb9"
format_stamp: "Formatted at 2026-08-11 12:09:38 on dist-test-slave-nj21"
server_key: "270095fed818a93119ebf68012fb92ec"
server_key_iv: "b34ea5f7fb30104e37db2f8649d0dec8"
server_key_version: "encryptionkey@0"
I20260811 12:09:38.871227 13438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:38.884649 13438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:38.885313 13438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:38.885571 13438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:38.902498 13438 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:42739
I20260811 12:09:38.902531 13550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:42739 every 8 connection(s)
I20260811 12:09:38.903752 13438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:38.907229 13551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:38.908944 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13438
I20260811 12:09:38.909163 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:38.910019 4291 external_mini_cluster.cc:1428] Setting key 0d2abfd4f232831b33c1dcaa38d1b8c6
I20260811 12:09:38.919266 13551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9: Bootstrap starting.
I20260811 12:09:38.922441 13551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:38.923207 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:38.911874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56892 (local address 127.4.48.254:42739)
0811 12:09:38.912864 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:38.912877 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:38.912922 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:09:38.914541 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:38.914564 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:38.914979 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:38.915271 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:38.915888 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.917399 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.918776 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:38.919469 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:38.919635 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:38.920790 (+ 1155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:38.920798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:38.920810 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:38.921316 (+ 506us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:38.921410 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:38.921431 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:38.921528 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:38.921759 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 12:09:38.922169 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":373,"threads_started":1}
I20260811 12:09:38.923549 13551 log.cc:824] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:38.926319 13551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9: No bootstrap required, opened a new log
I20260811 12:09:38.926695 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:09:38.933693 13551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } }
I20260811 12:09:38.934010 13551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:38.934098 13551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1673bbad67bd44eeabc84f3efad37bb9, State: Initialized, Role: FOLLOWER
I20260811 12:09:38.934599 13551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } }
I20260811 12:09:38.934824 13551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:38.934945 13551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:38.935093 13551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:38.937369 13551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } }
I20260811 12:09:38.937829 13551 leader_election.cc:302] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1673bbad67bd44eeabc84f3efad37bb9; no voters:
I20260811 12:09:38.938359 13551 leader_election.cc:288] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:38.938481 13560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:38.939122 13560 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [term 1 LEADER]: Becoming Leader. State: Replica: 1673bbad67bd44eeabc84f3efad37bb9, State: Running, Role: LEADER
I20260811 12:09:38.939694 13560 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } }
I20260811 12:09:38.940372 13551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:38.942627 13568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1673bbad67bd44eeabc84f3efad37bb9. Latest consensus state: current_term: 1 leader_uuid: "1673bbad67bd44eeabc84f3efad37bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } } }
I20260811 12:09:38.942588 13567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1673bbad67bd44eeabc84f3efad37bb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1673bbad67bd44eeabc84f3efad37bb9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42739 } } }
I20260811 12:09:38.943008 13567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:38.943008 13568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:38.944551 13575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:38.949236 13575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:38.958813 13575 catalog_manager.cc:1347] Generated new cluster ID: 5a52c780f7574919868938c5cf5c2759
I20260811 12:09:38.958913 13575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:38.994807 13575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:38.996069 13575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:39.005944 13575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1673bbad67bd44eeabc84f3efad37bb9: Generated new TSK 0
I20260811 12:09:39.006650 13575 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:09:39.029036 13555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:39.020956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56900 (local address 127.4.48.254:42739)
0811 12:09:39.021143 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:39.021150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:39.021543 (+ 393us) server_negotiation.cc:407] Connection header received
0811 12:09:39.021750 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:39.021756 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:39.021854 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:39.022001 (+ 147us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:09:39.022395 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.023484 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.024804 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.025507 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.025634 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:39.026098 (+ 464us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:09:39.028414 (+ 2316us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36509/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:09:39.028728 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36509/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 12:09:39.029160 13555 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56900: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36509/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 12:09:39.030661 13593 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:39.020834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42739 (local address 127.0.0.1:56900)
0811 12:09:39.021453 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:39.021472 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:39.021592 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:39.021999 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:39.022002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:39.022010 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:09:39.022262 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:39.022270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.023598 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.023602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:39.024669 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:39.024681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.025676 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.025680 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:39.025810 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:39.025844 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:09:39.028626 (+ 2782us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36509/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 12:09:39.030253 (+ 1627us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.48.254:42739: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36509/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":497,"mutex_wait_us":1349,"thread_start_us":333,"threads_started":1}
I20260811 12:09:39.031613 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13438
I20260811 12:09:39.032733 13545 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 12:09:39.040426 13594 webserver.cc:533] Webserver: sslize failed:
E20260811 12:09:39.040510 13594 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 12:09:39.123536 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13438
2026-08-11T12:09:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (715 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T12:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:39Z Disabled control of system clock
I20260811 12:09:39.214267 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38197
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:46385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38197
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/testchainca.cert with env {}
W20260811 12:09:39.538748 13601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:39.539033 13601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:39.539157 13601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:39.546154 13601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:39.546248 13601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:39.546321 13601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:39.546397 13601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:39.546454 13601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 12:09:39.546535 13601 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 12:09:39.546587 13601 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 12:09:39.556152 13601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:46385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38197
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:39.557515 13601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:39.558753 13601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:39.569908 13611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:39.570075 13616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:39.570107 13612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:39.571015 13601 server_base.cc:1034] running on GCE node
I20260811 12:09:39.571756 13601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:39.573045 13601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:39.574241 13601 hybrid_clock.cc:648] HybridClock initialized: now 1786450179574198 us; error 30 us; skew 500 ppm
I20260811 12:09:39.574558 13601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:09:39.576642 13601 webserver.cc:460] Webserver started at http://127.4.48.254:45395/ using document root <none> and password file <none>
I20260811 12:09:39.577291 13601 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:39.577395 13601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:39.577685 13601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:39.580456 13601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "3522d5bd316a480b89cce36d84dfbabd"
format_stamp: "Formatted at 2026-08-11 12:09:39 on dist-test-slave-nj21"
server_key: "0ab3f471fb1830625cf0018104dd39e7"
server_key_iv: "12a394fe0daa8c03dd5f1ed2fc708523"
server_key_version: "encryptionkey@0"
I20260811 12:09:39.581199 13601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "3522d5bd316a480b89cce36d84dfbabd"
format_stamp: "Formatted at 2026-08-11 12:09:39 on dist-test-slave-nj21"
server_key: "0ab3f471fb1830625cf0018104dd39e7"
server_key_iv: "12a394fe0daa8c03dd5f1ed2fc708523"
server_key_version: "encryptionkey@0"
I20260811 12:09:39.585959 13601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:39.589201 13625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:39.590391 13601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:09:39.590554 13601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "3522d5bd316a480b89cce36d84dfbabd"
format_stamp: "Formatted at 2026-08-11 12:09:39 on dist-test-slave-nj21"
server_key: "0ab3f471fb1830625cf0018104dd39e7"
server_key_iv: "12a394fe0daa8c03dd5f1ed2fc708523"
server_key_version: "encryptionkey@0"
I20260811 12:09:39.590737 13601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:39.604403 13601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:39.605104 13601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:39.605360 13601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:39.622430 13601 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:38197
I20260811 12:09:39.622465 13723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:38197 every 8 connection(s)
I20260811 12:09:39.623698 13601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:39.627498 13725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:39.632494 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13601
I20260811 12:09:39.632694 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:39.633494 4291 external_mini_cluster.cc:1428] Setting key 2099de5bd1321a4876da2bab2ef713cd
I20260811 12:09:39.638770 13725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd: Bootstrap starting.
I20260811 12:09:39.641788 13725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:39.642797 13725 log.cc:824] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:39.645547 13725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd: No bootstrap required, opened a new log
I20260811 12:09:39.645902 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:39.635471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.4.48.254:38197)
0811 12:09:39.636415 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:39.636426 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:39.636467 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:09:39.638108 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:39.638131 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:39.638544 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:39.638888 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:39.639402 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.640720 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.641945 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.642590 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.642767 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:39.643821 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:39.643829 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:39.643845 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:09:39.644326 (+ 481us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:39.644398 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:09:39.644419 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:39.644526 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:39.644733 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:09:39.645013 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":332,"threads_started":1}
I20260811 12:09:39.649317 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:09:39.653095 13725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } }
I20260811 12:09:39.653419 13725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:39.653512 13725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3522d5bd316a480b89cce36d84dfbabd, State: Initialized, Role: FOLLOWER
I20260811 12:09:39.654004 13725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } }
I20260811 12:09:39.654224 13725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:39.654347 13725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:39.654506 13725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:39.656658 13725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } }
I20260811 12:09:39.657136 13725 leader_election.cc:302] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3522d5bd316a480b89cce36d84dfbabd; no voters:
I20260811 12:09:39.657660 13725 leader_election.cc:288] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:39.657783 13730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:39.658874 13730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [term 1 LEADER]: Becoming Leader. State: Replica: 3522d5bd316a480b89cce36d84dfbabd, State: Running, Role: LEADER
I20260811 12:09:39.659319 13730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } }
I20260811 12:09:39.659700 13725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:39.661867 13731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3522d5bd316a480b89cce36d84dfbabd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } } }
I20260811 12:09:39.662487 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3522d5bd316a480b89cce36d84dfbabd. Latest consensus state: current_term: 1 leader_uuid: "3522d5bd316a480b89cce36d84dfbabd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3522d5bd316a480b89cce36d84dfbabd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38197 } } }
I20260811 12:09:39.662745 13731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:39.662824 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:39.664351 13741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:39.668845 13741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:39.676944 13741 catalog_manager.cc:1347] Generated new cluster ID: 3caa08904c744d6eb2e5d862ca98e2ea
I20260811 12:09:39.677156 13741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:39.697610 13741 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:39.698773 13741 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:39.721004 13741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3522d5bd316a480b89cce36d84dfbabd: Generated new TSK 0
I20260811 12:09:39.721798 13741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:09:40.292806 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:39.737412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40284 (local address 127.4.48.254:38197)
0811 12:09:39.737585 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:39.737594 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:39.738104 (+ 510us) server_negotiation.cc:407] Connection header received
0811 12:09:39.738282 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:39.738288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:39.738380 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:39.738544 (+ 164us) server_negotiation.cc:226] Negotiated authn=JWT
0811 12:09:39.738988 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.740164 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.741485 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.742167 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.742300 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:39.742698 (+ 398us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 12:09:40.292212 (+549514us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:09:40.292567 (+ 355us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40284: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 12:09:40.292940 13728 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40284: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 12:09:40.293118 13763 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:39.737406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38197 (local address 127.0.0.1:40284)
0811 12:09:39.738011 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:39.738026 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:39.738142 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:39.738539 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:39.738542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:39.738551 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 12:09:39.738839 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:39.738848 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.740285 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.740288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:39.741354 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:39.741366 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:39.742307 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:39.742311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:39.742419 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:39.742450 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 12:09:40.292507 (+550057us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 12:09:40.292763 (+ 256us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.4.48.254:38197: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":326,"threads_started":1}
I20260811 12:09:40.294235 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13601
I20260811 12:09:40.295300 13715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:40.393522 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13601
2026-08-11T12:09:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1252 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (411 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (408 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:41 dist-test-slave-nj21 krb5kdc[13787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:41 dist-test-slave-nj21 krb5kdc[13787](info): set up 2 sockets
Aug 11 12:09:41 dist-test-slave-nj21 krb5kdc[13787](info): commencing operation
krb5kdc: starting...
W20260811 12:09:42.431454 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:42 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:09:42.641122 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46333
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46333
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:42.975366 13807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:42.975641 13807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:42.975798 13807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:42.982677 13807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:42.982786 13807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:42.982858 13807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:42.982932 13807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:42.982987 13807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:42.992329 13807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46333
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46333
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:42.993476 13807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:42.994679 13807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:43.005766 13820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:43.005839 13824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:43.005872 13821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:43.006582 13807 server_base.cc:1034] running on GCE node
I20260811 12:09:43.007261 13807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:43.008548 13807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:43.009745 13807 hybrid_clock.cc:648] HybridClock initialized: now 1786450183009695 us; error 34 us; skew 500 ppm
I20260811 12:09:43.010052 13807 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:43 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450183, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:43.012949 13807 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:09:43.014258 13807 webserver.cc:460] Webserver started at http://127.4.48.254:38231/ using document root <none> and password file <none>
I20260811 12:09:43.014844 13807 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:43.014940 13807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:43.015187 13807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:43.017956 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "8d414ac6a13d481091a5ac4fedf1c1e1"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "324d05e641e393e738f5b4160ada9bb1"
server_key_iv: "d0708044d98a29d81e0a38def880cba2"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.018883 13807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "8d414ac6a13d481091a5ac4fedf1c1e1"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "324d05e641e393e738f5b4160ada9bb1"
server_key_iv: "d0708044d98a29d81e0a38def880cba2"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.024180 13807 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:09:43.027572 13833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:43.028874 13807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:09:43.029050 13807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "8d414ac6a13d481091a5ac4fedf1c1e1"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "324d05e641e393e738f5b4160ada9bb1"
server_key_iv: "d0708044d98a29d81e0a38def880cba2"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.029237 13807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:43.047837 13807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:43.051641 13807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:43.051965 13807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:43.070079 13807 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:46333
I20260811 12:09:43.070111 13927 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:46333 every 8 connection(s)
I20260811 12:09:43.071431 13807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:43.075110 13928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:43.080917 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13807
I20260811 12:09:43.081143 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:43.081971 4291 external_mini_cluster.cc:1428] Setting key 18672fcc6bc9b9cd12df9e3c20f0b19b
I20260811 12:09:43.085994 13928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1: Bootstrap starting.
I20260811 12:09:43.089529 13928 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:43.090570 13928 log.cc:824] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:43.093490 13928 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1: No bootstrap required, opened a new log
Aug 11 12:09:43 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:43.102082 13928 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } }
I20260811 12:09:43.102420 13928 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:43.102521 13928 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d414ac6a13d481091a5ac4fedf1c1e1, State: Initialized, Role: FOLLOWER
I20260811 12:09:43.103129 13928 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } }
I20260811 12:09:43.103380 13928 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:43.103549 13928 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:43.103780 13928 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:43.104684 13938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:43.084116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56082 (local address 127.4.48.254:46333)
0811 12:09:43.084939 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:43.084950 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:43.084994 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:09:43.086823 (+ 1829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:43.086847 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:43.087203 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:43.087537 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:43.089356 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.090743 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.092233 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.092886 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.093015 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:43.097705 (+ 4690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:43.097713 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:43.097727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:43.097762 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:43.100767 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:43.101430 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:43.101437 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:43.101443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:43.101601 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:43.101985 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:43.101989 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:43.101991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:43.102525 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:43.102828 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:43.103243 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 12:09:43.103634 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":368,"threads_started":1}
I20260811 12:09:43.106122 13928 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } }
I20260811 12:09:43.106666 13928 leader_election.cc:302] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d414ac6a13d481091a5ac4fedf1c1e1; no voters:
I20260811 12:09:43.107283 13928 leader_election.cc:288] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:43.107410 13942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:43.108104 13942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [term 1 LEADER]: Becoming Leader. State: Replica: 8d414ac6a13d481091a5ac4fedf1c1e1, State: Running, Role: LEADER
I20260811 12:09:43.108691 13942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } }
I20260811 12:09:43.109402 13928 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:43.111474 13946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d414ac6a13d481091a5ac4fedf1c1e1. Latest consensus state: current_term: 1 leader_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } } }
I20260811 12:09:43.111533 13945 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d414ac6a13d481091a5ac4fedf1c1e1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46333 } } }
I20260811 12:09:43.111935 13945 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:43.111935 13946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:43.116290 13954 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:43.121356 13954 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:43.129617 13954 catalog_manager.cc:1347] Generated new cluster ID: 80397c81aedf426abcf7ad03a2bbdeec
I20260811 12:09:43.129719 13954 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:43.150731 13954 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:43.152032 13954 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:43.159906 13954 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d414ac6a13d481091a5ac4fedf1c1e1: Generated new TSK 0
I20260811 12:09:43.160641 13954 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:09:43.214836 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:43.536810 13973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:43.537061 13973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:43.537195 13973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:43.544232 13973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:43.544343 13973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:43.544497 13973 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:43.553707 13973 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:43.554890 13973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:43.556237 13973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:43.568920 13978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:43.568893 13979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:43.569303 13982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:43.570577 13973 server_base.cc:1034] running on GCE node
I20260811 12:09:43.571123 13973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:43.572028 13973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:43.573227 13973 hybrid_clock.cc:648] HybridClock initialized: now 1786450183573181 us; error 39 us; skew 500 ppm
I20260811 12:09:43.573518 13973 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:43 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450183, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:43.576438 13973 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:09:43.577718 13973 webserver.cc:460] Webserver started at http://127.4.48.193:39617/ using document root <none> and password file <none>
I20260811 12:09:43.578383 13973 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:43.578501 13973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:43.578790 13973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:43.581627 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "0a65b4d7bbbc4084a944c06590abbaa7"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "0f960c74f80d2dcdf58aed424b0bd26c"
server_key_iv: "046009f8faad48c32fe84d918e6b614f"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.582394 13973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "0a65b4d7bbbc4084a944c06590abbaa7"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "0f960c74f80d2dcdf58aed424b0bd26c"
server_key_iv: "046009f8faad48c32fe84d918e6b614f"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.587339 13973 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 12:09:43.590607 13996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:43.591836 13973 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:09:43.591989 13973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "0a65b4d7bbbc4084a944c06590abbaa7"
format_stamp: "Formatted at 2026-08-11 12:09:43 on dist-test-slave-nj21"
server_key: "0f960c74f80d2dcdf58aed424b0bd26c"
server_key_iv: "046009f8faad48c32fe84d918e6b614f"
server_key_version: "encryptionkey@0"
I20260811 12:09:43.592166 13973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:43.604122 13973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:43.607658 13973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:43.607957 13973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:43.608839 13973 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:43.610138 13973 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:43.610219 13973 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:43.610342 13973 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:43.610407 13973 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:43.640197 13973 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:36671
I20260811 12:09:43.640240 14201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:36671 every 8 connection(s)
I20260811 12:09:43.641441 13973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:43.644778 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 13973
I20260811 12:09:43.644996 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:43.645807 4291 external_mini_cluster.cc:1428] Setting key 25bc265ed22707e7dfa0c7686121f846
Aug 11 12:09:43 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450183, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:43.661489 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:43.643859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:46933 (local address 127.4.48.254:46333)
0811 12:09:43.644307 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:43.644311 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:43.645413 (+ 1102us) server_negotiation.cc:407] Connection header received
0811 12:09:43.647452 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:43.647458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:43.647540 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:43.647634 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:43.649146 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.650197 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.651717 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.652254 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.652383 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:43.655756 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:43.655763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:43.655767 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:43.655805 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:43.658334 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:43.659079 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:43.659084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:43.659088 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:43.659252 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:43.659728 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:43.659732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:43.659734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:43.660056 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:43.660219 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:43.661050 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 12:09:43.661221 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":210,"threads_started":1}
I20260811 12:09:43.662743 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:43.644426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46333 (local address 127.4.48.193:46933)
0811 12:09:43.645257 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:43.645302 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:43.647148 (+ 1846us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:43.647938 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:43.647945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:43.648435 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:43.648973 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:43.648991 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.650336 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.650340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:43.651559 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:43.651571 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:43.652451 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:43.652456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:43.652643 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:43.653238 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:43.653260 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:43.655524 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:43.658447 (+ 2923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:43.658455 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:43.658468 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:43.658981 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:43.659361 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:43.659366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:43.659369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:43.659596 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:43.660203 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:43.660210 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:43.660661 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 12:09:43.660944 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 12:09:43.661429 (+ 485us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":370,"threads_started":1}
I20260811 12:09:43.664245 14203 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46333
I20260811 12:09:43.664530 14203 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:43.665230 14203 heartbeater.cc:511] Master 127.4.48.254:46333 requested a full tablet report, sending...
I20260811 12:09:43.667182 13864 ts_manager.cc:194] Registered new tserver with Master: 0a65b4d7bbbc4084a944c06590abbaa7 (127.4.48.193:36671)
I20260811 12:09:43.668601 13864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:46933
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:09:43.702634 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:44.023949 14211 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:44.024189 14211 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:44.024286 14211 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:44.031213 14211 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:44.031308 14211 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:44.031420 14211 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:44.040596 14211 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:44.041788 14211 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:44.043174 14211 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:44.055619 14219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:44.055589 14220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:44.055534 14223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:44.056560 14211 server_base.cc:1034] running on GCE node
I20260811 12:09:44.057154 14211 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:44.058022 14211 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:44.059212 14211 hybrid_clock.cc:648] HybridClock initialized: now 1786450184059166 us; error 36 us; skew 500 ppm
I20260811 12:09:44.059497 14211 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.062438 14211 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:09:44.063773 14211 webserver.cc:460] Webserver started at http://127.4.48.194:46525/ using document root <none> and password file <none>
I20260811 12:09:44.064406 14211 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:44.064505 14211 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:44.064764 14211 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:44.067514 14211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "ce8643ae87284f6c810ef6adc2c41205"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "c2680004495617a1a137b1acde3c4c4f"
server_key_iv: "e3147db7dbc3a90dd659791bb0f34b14"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.068292 14211 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "ce8643ae87284f6c810ef6adc2c41205"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "c2680004495617a1a137b1acde3c4c4f"
server_key_iv: "e3147db7dbc3a90dd659791bb0f34b14"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.073412 14211 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:09:44.076748 14235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.077980 14211 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:09:44.078127 14211 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "ce8643ae87284f6c810ef6adc2c41205"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "c2680004495617a1a137b1acde3c4c4f"
server_key_iv: "e3147db7dbc3a90dd659791bb0f34b14"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.078295 14211 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:44.101645 14211 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:44.105159 14211 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:44.105417 14211 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:44.106303 14211 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:44.107534 14211 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:44.107611 14211 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.107746 14211 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:44.107801 14211 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.137277 14211 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:37757
I20260811 12:09:44.137337 14423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:37757 every 8 connection(s)
I20260811 12:09:44.138509 14211 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:44.142933 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 14211
I20260811 12:09:44.143121 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:44.143705 4291 external_mini_cluster.cc:1428] Setting key e8422a2e637c3d8b8b1d9b86f4166665
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:44.159554 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.141044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:44599 (local address 127.4.48.254:46333)
0811 12:09:44.141258 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.141264 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.142394 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 12:09:44.144683 (+ 2289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.144689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.144776 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.144870 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.146725 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.147441 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.149520 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.149936 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.150028 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.153888 (+ 3860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.153893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.153896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.153929 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.156650 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.157339 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.157344 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.157347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.157507 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.157876 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.157881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.157884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.158192 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.158334 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.158997 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.159245 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:09:44.160512 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.141543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46333 (local address 127.4.48.194:44599)
0811 12:09:44.142246 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:44.142283 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:44.144447 (+ 2164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:44.145131 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:44.145141 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:44.145771 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:44.146555 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:44.146581 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.147595 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.147602 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:44.149381 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:44.149395 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.150146 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.150152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:44.150427 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.151103 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:44.151131 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:44.153666 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:44.156778 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:44.156785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:44.156797 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:44.157217 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:44.157601 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:44.157604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:44.157606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:44.157771 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:44.158294 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:44.158300 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:44.158607 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 12:09:44.158829 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 12:09:44.159205 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":284,"threads_started":1}
I20260811 12:09:44.161993 14425 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46333
I20260811 12:09:44.162295 14425 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:44.163024 14425 heartbeater.cc:511] Master 127.4.48.254:46333 requested a full tablet report, sending...
I20260811 12:09:44.164701 13864 ts_manager.cc:194] Registered new tserver with Master: ce8643ae87284f6c810ef6adc2c41205 (127.4.48.194:37757)
I20260811 12:09:44.165879 13864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:44599
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:09:44.196089 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:44.513638 14435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:44.513876 14435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:44.513972 14435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:44.520911 14435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:44.521004 14435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:44.521116 14435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:44.530086 14435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46333
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:44.531177 14435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:44.532418 14435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:44.544567 14456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:44.544637 14449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:44.544615 14445 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:44.545507 14435 server_base.cc:1034] running on GCE node
I20260811 12:09:44.546051 14435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:44.546876 14435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:44.548063 14435 hybrid_clock.cc:648] HybridClock initialized: now 1786450184548011 us; error 39 us; skew 500 ppm
I20260811 12:09:44.548348 14435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.551210 14435 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:09:44.552560 14435 webserver.cc:460] Webserver started at http://127.4.48.195:40725/ using document root <none> and password file <none>
I20260811 12:09:44.553174 14435 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:44.553296 14435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:44.553579 14435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:44.556344 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "4ebe8bcc4eba420bb7cc180776e83b28"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "a7567f2c3ace83506b49b30f38dd5ae0"
server_key_iv: "c596bec408f4c65382373d30563eb86b"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.557195 14435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "4ebe8bcc4eba420bb7cc180776e83b28"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "a7567f2c3ace83506b49b30f38dd5ae0"
server_key_iv: "c596bec408f4c65382373d30563eb86b"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.562036 14435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:09:44.565596 14468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.566787 14435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:09:44.566952 14435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "4ebe8bcc4eba420bb7cc180776e83b28"
format_stamp: "Formatted at 2026-08-11 12:09:44 on dist-test-slave-nj21"
server_key: "a7567f2c3ace83506b49b30f38dd5ae0"
server_key_iv: "c596bec408f4c65382373d30563eb86b"
server_key_version: "encryptionkey@0"
I20260811 12:09:44.567129 14435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:44.582652 14435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:44.586627 14435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:44.586894 14435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:44.587821 14435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:44.589088 14435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:44.589159 14435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.589272 14435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:44.589322 14435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:44.618386 14435 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:35459
I20260811 12:09:44.618448 14658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:35459 every 8 connection(s)
I20260811 12:09:44.619585 14435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:44.623132 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 14435
I20260811 12:09:44.623309 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:44.623816 4291 external_mini_cluster.cc:1428] Setting key 8d7c550610e4a97a4163992512f770ca
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:09:44.638052 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.621907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:51053 (local address 127.4.48.254:46333)
0811 12:09:44.622127 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.622133 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.623220 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 12:09:44.625188 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.625193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.625260 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.625332 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.627037 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.627823 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.629243 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.629642 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.629730 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.632916 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.632921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.632924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.632956 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.635208 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.635895 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.635900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.635904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.636020 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.636431 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.636435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.636437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.636706 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.636837 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.637525 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.637715 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:09:44.639081 14667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.622363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46333 (local address 127.4.48.195:51053)
0811 12:09:44.623065 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:44.623111 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:44.624903 (+ 1792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:44.625611 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:44.625620 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:44.626204 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:44.626867 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:44.626896 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.627949 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.627953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:44.629138 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:44.629150 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.629800 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.629804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:44.629991 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.630574 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:44.630596 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:44.632722 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:44.635333 (+ 2611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:44.635341 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:44.635355 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:44.635788 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:44.636120 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:44.636125 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:44.636129 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:44.636312 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:44.636828 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:44.636836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:44.637136 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 12:09:44.637389 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 12:09:44.637826 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":295,"threads_started":1}
I20260811 12:09:44.640342 14661 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46333
I20260811 12:09:44.640610 14661 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:44.641299 14661 heartbeater.cc:511] Master 127.4.48.254:46333 requested a full tablet report, sending...
I20260811 12:09:44.642653 13864 ts_manager.cc:194] Registered new tserver with Master: 4ebe8bcc4eba420bb7cc180776e83b28 (127.4.48.195:35459)
I20260811 12:09:44.643445 13864 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:51053
I20260811 12:09:44.650943 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:09:44.671350 14203 heartbeater.cc:503] Master 127.4.48.254:46333 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/test-admin.keytab.
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.680516 4291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 12:09:44.685271 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:44.683011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58676 (local address 127.4.48.193:36671)
0811 12:09:44.683652 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.683658 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.683714 (+ 56us) server_negotiation.cc:407] Connection header received
0811 12:09:44.683771 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.683778 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.683989 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.684107 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.684928 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58676: BlockingRecv error: recv got EOF from 127.0.0.1:58676 (error 108)
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":347,"threads_started":1}
W20260811 12:09:44.685339 4291 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.687011 4291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.688239 4291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:44.702091 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.689738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.4.48.193:36671)
0811 12:09:44.689922 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.689927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.690121 (+ 194us) server_negotiation.cc:407] Connection header received
0811 12:09:44.690229 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.690233 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.690285 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.690362 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.691215 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.692197 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.693318 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.693963 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.694058 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.697406 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.697417 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.697431 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.697467 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.699908 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.700456 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.700462 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.700464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.700575 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.700841 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.700844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.700846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.701168 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.701382 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.701599 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.701842 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:09:44.707089 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:44.705726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.4.48.193:36671)
0811 12:09:44.705919 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.705923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.706017 (+ 94us) server_negotiation.cc:407] Connection header received
0811 12:09:44.706124 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.706127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.706188 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.706257 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.706874 (+ 617us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58700: BlockingRecv error: recv got EOF from 127.0.0.1:58700 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 12:09:44.707268 4291 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc' of type FILE
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.708597 4291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.709875 4291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:44.722206 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.710999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58706 (local address 127.4.48.193:36671)
0811 12:09:44.711167 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.711172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.711407 (+ 235us) server_negotiation.cc:407] Connection header received
0811 12:09:44.711501 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.711504 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.711551 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.711618 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.712647 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.713323 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.714411 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.714786 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.714887 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.717988 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.717993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.717995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.718037 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.720317 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.720822 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.720825 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.720828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.720950 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.721219 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.721222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.721224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.721450 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.721628 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.721798 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.721944 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 12:09:44.725859 14673 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:09:44.724414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.4.48.193:36671)
0811 12:09:44.724586 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.724591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.724756 (+ 165us) server_negotiation.cc:407] Connection header received
0811 12:09:44.724870 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.724874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.724934 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.725005 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.725677 (+ 672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51940: BlockingRecv error: recv got EOF from 127.0.0.1:51940 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 12:09:44.726426 4291 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786450132843697-4291-0/krb5kdc/krb5cc)
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.727780 4291 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:44.728926 4291 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:09:44.741196 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.730073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.4.48.193:36671)
0811 12:09:44.730252 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.730257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.730427 (+ 170us) server_negotiation.cc:407] Connection header received
0811 12:09:44.730522 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.730525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.730571 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.730637 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.731482 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.732219 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.733393 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.733773 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.733858 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.736855 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.736860 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.736871 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.736899 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.739200 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.739731 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.739735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.739737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.739884 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.740160 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.740163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.740165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.740378 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.740570 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.740754 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.740904 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:09:44.742496 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13973
I20260811 12:09:44.752425 14673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.743088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.4.48.193:36671)
0811 12:09:44.743221 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.743225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.743523 (+ 298us) server_negotiation.cc:407] Connection header received
0811 12:09:44.743621 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.743625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.743699 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.743775 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.744581 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.745197 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.746281 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.746692 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.746780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.748449 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.748453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.748455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.748480 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.750514 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.751039 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.751043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.751045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.751161 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.751417 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.751420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.751422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.751632 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.751855 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.752057 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.752194 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 12:09:44.752907 14195 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:44.834004 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13973
I20260811 12:09:44.857690 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 14211
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:09:44.871901 14694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.858565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46996 (local address 127.4.48.194:37757)
0811 12:09:44.859090 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.859097 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.859123 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:09:44.859181 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.859188 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.859405 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.859530 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.860505 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.861541 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.862696 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.863230 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.863336 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.866830 (+ 3494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.866841 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.866855 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.866888 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.869351 (+ 2463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.869978 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.869988 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.869991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.870106 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.870386 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.870389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.870391 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.870764 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.871093 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.871259 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.871530 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":257,"threads_started":1}
I20260811 12:09:44.873121 14413 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:44.953976 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 14211
I20260811 12:09:44.977406 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 14435
Aug 11 12:09:44 dist-test-slave-nj21 krb5kdc[13787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:09:44.992084 14697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:44.978321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56172 (local address 127.4.48.195:35459)
0811 12:09:44.978942 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:44.978949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:44.978978 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:09:44.979042 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:44.979048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:44.979254 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:44.979385 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:44.980489 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.981509 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.982710 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:44.983274 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:44.983371 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:44.987128 (+ 3757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:44.987139 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:44.987154 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:44.987184 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:44.989675 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.990289 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.990295 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.990297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.990417 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:44.990689 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:44.990692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:44.990694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:44.991047 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:44.991266 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:44.991507 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 12:09:44.991757 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":304,"threads_started":1}
I20260811 12:09:44.993332 14648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:45.073948 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 14435
I20260811 12:09:45.094187 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 13807
I20260811 12:09:45.095306 13919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:45.196476 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 13807
2026-08-11T12:09:45Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3991 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:45 dist-test-slave-nj21 krb5kdc[14702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:45 dist-test-slave-nj21 krb5kdc[14702](info): set up 2 sockets
Aug 11 12:09:45 dist-test-slave-nj21 krb5kdc[14702](info): commencing operation
krb5kdc: starting...
W20260811 12:09:46.423061 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:46 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:46Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab.
Entry for principal oryx/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab.
I20260811 12:09:46.639027 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:33371
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab
--principal=oryx/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:33371
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:46.957608 14723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:46.957907 14723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:46.958045 14723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:46.964964 14723 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:46.965070 14723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:46.965147 14723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:46.965224 14723 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:46.965287 14723 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:46.975374 14723 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:33371
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab
--principal=oryx/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:33371
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:46.976747 14723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:46.978003 14723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:46.993254 14739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:46.993292 14738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:46.993661 14743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:46.994418 14723 server_base.cc:1034] running on GCE node
I20260811 12:09:46.995169 14723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:46.996624 14723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:46.997861 14723 hybrid_clock.cc:648] HybridClock initialized: now 1786450186997800 us; error 50 us; skew 500 ppm
I20260811 12:09:46.998179 14723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:46 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:47.001262 14723 init.cc:377] Logged in from keytab as oryx/127.4.48.254@KRBTEST.COM (short username oryx)
I20260811 12:09:47.002617 14723 webserver.cc:460] Webserver started at http://127.4.48.254:43999/ using document root <none> and password file <none>
I20260811 12:09:47.003275 14723 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:47.003404 14723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:47.003708 14723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:47.006592 14723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "6c85c904d2514636985e638d1f662f20"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "d94825f11f26fa63c4126d1a0646ed43"
server_key_iv: "84ad96d7b9a98b7e30178e75fe194297"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.007432 14723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "6c85c904d2514636985e638d1f662f20"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "d94825f11f26fa63c4126d1a0646ed43"
server_key_iv: "84ad96d7b9a98b7e30178e75fe194297"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.012491 14723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:09:47.015738 14756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:47.016950 14723 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 12:09:47.017113 14723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "6c85c904d2514636985e638d1f662f20"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "d94825f11f26fa63c4126d1a0646ed43"
server_key_iv: "84ad96d7b9a98b7e30178e75fe194297"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.017285 14723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:47.047099 14723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:47.050855 14723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:47.051116 14723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:47.068429 14723 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:33371
I20260811 12:09:47.068461 14849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:33371 every 8 connection(s)
I20260811 12:09:47.069728 14723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:47.073544 14850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:47.078262 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 14723
I20260811 12:09:47.078461 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:47.079114 4291 external_mini_cluster.cc:1428] Setting key f3620fdb350cd049ee3847302c6cc769
I20260811 12:09:47.084425 14850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Bootstrap starting.
I20260811 12:09:47.087424 14850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:47.088407 14850 log.cc:824] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:47.091759 14850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: No bootstrap required, opened a new log
Aug 11 12:09:47 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:47.100716 14850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } }
I20260811 12:09:47.101084 14850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:47.101178 14850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c85c904d2514636985e638d1f662f20, State: Initialized, Role: FOLLOWER
I20260811 12:09:47.101526 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:47.081151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60808 (local address 127.4.48.254:33371)
0811 12:09:47.082010 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:47.082021 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:47.082065 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:09:47.083753 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:47.083781 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:47.084151 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:47.084498 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:47.085947 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.087218 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.088560 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.089538 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.089718 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:47.094420 (+ 4702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:47.094428 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:47.094443 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:47.094480 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:47.097497 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:47.098277 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:47.098283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:47.098288 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:47.098444 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:47.098834 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:47.098837 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:47.098839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:47.099355 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:47.099593 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:47.100051 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 12:09:47.100381 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":384,"threads_started":1}
I20260811 12:09:47.101749 14850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } }
I20260811 12:09:47.101992 14850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:47.102136 14850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:47.102298 14850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:47.105057 14850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } }
I20260811 12:09:47.105520 14850 leader_election.cc:302] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6c85c904d2514636985e638d1f662f20; no voters:
I20260811 12:09:47.106035 14850 leader_election.cc:288] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:47.106153 14856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:47.107575 14856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 LEADER]: Becoming Leader. State: Replica: 6c85c904d2514636985e638d1f662f20, State: Running, Role: LEADER
I20260811 12:09:47.107961 14850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:47.108078 14856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } }
I20260811 12:09:47.110610 14858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c85c904d2514636985e638d1f662f20. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } }
I20260811 12:09:47.110951 14858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:47.111470 14857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } }
I20260811 12:09:47.111816 14857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:47.111974 14868 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:47.117285 14868 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:47.126356 14868 catalog_manager.cc:1347] Generated new cluster ID: f90e8db32f08402080f23baa79fe4c52
I20260811 12:09:47.126458 14868 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:47.143347 14868 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:47.145186 14868 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:47.152326 14868 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Generated new TSK 0
I20260811 12:09:47.153090 14868 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
I20260811 12:09:47.213418 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:47.532835 14893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:47.533088 14893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:47.533211 14893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:47.540187 14893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:47.540288 14893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:47.540438 14893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:47.549697 14893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:47.550894 14893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:47.552189 14893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:47.564687 14905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:47.564656 14917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:47.564623 14913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:47.565682 14893 server_base.cc:1034] running on GCE node
I20260811 12:09:47.566234 14893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:47.567142 14893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:47.568334 14893 hybrid_clock.cc:648] HybridClock initialized: now 1786450187568288 us; error 50 us; skew 500 ppm
I20260811 12:09:47.568619 14893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:47 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450187, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:47.571573 14893 init.cc:377] Logged in from keytab as oryx/127.4.48.193@KRBTEST.COM (short username oryx)
I20260811 12:09:47.572983 14893 webserver.cc:460] Webserver started at http://127.4.48.193:34973/ using document root <none> and password file <none>
I20260811 12:09:47.573630 14893 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:47.573745 14893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:47.574038 14893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:47.576942 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "0f9652be36c94c80b06d8c2d1d62f91e"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "143233e9bfa2646d4ce788a2d6c8177e"
server_key_iv: "a274768dc8caa077f65ae7b8dff85413"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.577659 14893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "0f9652be36c94c80b06d8c2d1d62f91e"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "143233e9bfa2646d4ce788a2d6c8177e"
server_key_iv: "a274768dc8caa077f65ae7b8dff85413"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.582523 14893 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:09:47.585865 14938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:47.587126 14893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:09:47.587266 14893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "0f9652be36c94c80b06d8c2d1d62f91e"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "143233e9bfa2646d4ce788a2d6c8177e"
server_key_iv: "a274768dc8caa077f65ae7b8dff85413"
server_key_version: "encryptionkey@0"
I20260811 12:09:47.587431 14893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:47.608099 14893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:47.612123 14893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:47.612396 14893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:47.613305 14893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:47.614583 14893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:47.614660 14893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:47.614778 14893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:47.614838 14893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:47.644897 14893 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35003
I20260811 12:09:47.644943 15132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35003 every 8 connection(s)
I20260811 12:09:47.646060 14893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:47.653069 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 14893
I20260811 12:09:47.653249 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:47.653920 4291 external_mini_cluster.cc:1428] Setting key 3e1819c395884e4766cda288fce23d54
Aug 11 12:09:47 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450187, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:47.665031 15143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:47.648570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:57669 (local address 127.4.48.254:33371)
0811 12:09:47.649011 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:47.649016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:47.649977 (+ 961us) server_negotiation.cc:407] Connection header received
0811 12:09:47.651761 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:47.651767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:47.651857 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:47.651952 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:47.653496 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.654339 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.656139 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.656548 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.656657 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:47.659862 (+ 3205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:47.659867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:47.659870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:47.659901 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:47.662190 (+ 2289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:47.662854 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:47.662859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:47.662862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:47.662977 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:47.663377 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:47.663382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:47.663386 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:47.663642 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:47.663806 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:47.664454 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 12:09:47.664682 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":235,"threads_started":1}
I20260811 12:09:47.665936 15146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:47.649107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:57669)
0811 12:09:47.649818 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:47.649861 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:47.651483 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:47.652194 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:47.652202 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:47.652703 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:47.653302 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:47.653330 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.654486 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.654490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:47.656005 (+ 1515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:47.656018 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:47.656762 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:47.656769 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:47.656979 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:47.657597 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:47.657623 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:47.659635 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:47.662324 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:47.662333 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:47.662347 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:47.662741 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:47.663078 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:47.663083 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:47.663087 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:47.663263 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:47.663770 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:47.663778 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:47.664092 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 12:09:47.664335 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 12:09:47.664824 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260811 12:09:47.667371 15136 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:47.667637 15136 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:47.668368 15136 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:47.670598 14787 ts_manager.cc:194] Registered new tserver with Master: 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003)
I20260811 12:09:47.672472 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:57669
WARNING: no policy specified for oryx/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
I20260811 12:09:47.707031 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:48.030043 15150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:48.030292 15150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:48.030390 15150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:48.037261 15150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:48.037360 15150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:48.037474 15150 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:48.046643 15150 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:48.047901 15150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:48.049219 15150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:48.062736 15161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:48.062853 15166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:48.062892 15163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:48.063489 15150 server_base.cc:1034] running on GCE node
I20260811 12:09:48.064112 15150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:48.064975 15150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:48.066174 15150 hybrid_clock.cc:648] HybridClock initialized: now 1786450188066130 us; error 33 us; skew 500 ppm
I20260811 12:09:48.066465 15150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:48 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450188, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:48.074761 15150 init.cc:377] Logged in from keytab as oryx/127.4.48.194@KRBTEST.COM (short username oryx)
I20260811 12:09:48.076402 15150 webserver.cc:460] Webserver started at http://127.4.48.194:43661/ using document root <none> and password file <none>
I20260811 12:09:48.077263 15150 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:48.077404 15150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:48.077711 15150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:48.081530 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "13fb3c5909d948cfb9c261038eed3f36"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "437444766b3d9c3b894eb24bff86e783"
server_key_iv: "87c8a318fc77928d543e6a94f7a06ce3"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.082459 15150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "13fb3c5909d948cfb9c261038eed3f36"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "437444766b3d9c3b894eb24bff86e783"
server_key_iv: "87c8a318fc77928d543e6a94f7a06ce3"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.088165 15150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 12:09:48.091441 15178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.092732 15150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:48.092866 15150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "13fb3c5909d948cfb9c261038eed3f36"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "437444766b3d9c3b894eb24bff86e783"
server_key_iv: "87c8a318fc77928d543e6a94f7a06ce3"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.093019 15150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:48.130805 15150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:48.134972 15150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:48.135238 15150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:48.136169 15150 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:48.137462 15150 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:48.137537 15150 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.137625 15150 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:48.137667 15150 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.166561 15150 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:41393
I20260811 12:09:48.166625 15396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:41393 every 8 connection(s)
I20260811 12:09:48.167881 15150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:48.170269 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 15150
I20260811 12:09:48.170459 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:48.170994 4291 external_mini_cluster.cc:1428] Setting key 695e6e5c4117b611a3649861d5accda9
Aug 11 12:09:48 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450188, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:48.189074 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:48.170321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:45039 (local address 127.4.48.254:33371)
0811 12:09:48.170858 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:48.170864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:48.171795 (+ 931us) server_negotiation.cc:407] Connection header received
0811 12:09:48.173918 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:48.173922 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:48.173986 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:48.174073 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:48.175942 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.176650 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.178609 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.179003 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.179095 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:48.182873 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:48.182878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:48.182881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:48.182915 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:48.186043 (+ 3128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:48.186816 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:48.186821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:48.186824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:48.186978 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:48.187367 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:48.187372 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:48.187376 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:48.187726 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:48.187862 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:48.188497 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 12:09:48.188711 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":278,"threads_started":1}
I20260811 12:09:48.189769 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:48.170802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:45039)
0811 12:09:48.171617 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:48.171657 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:48.173632 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:48.174324 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:48.174334 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:48.175008 (+ 674us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:48.175791 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:48.175814 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.176802 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.176807 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:48.178479 (+ 1672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:48.178493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.179224 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.179231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:48.179503 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:48.180171 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:48.180198 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:48.182651 (+ 2453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:48.186191 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:48.186200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:48.186215 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:48.186678 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:48.187071 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:48.187074 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:48.187076 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:48.187253 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:48.187834 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:48.187840 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:48.188126 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 12:09:48.188343 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 12:09:48.188694 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":313,"threads_started":1}
I20260811 12:09:48.191191 15397 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:48.191510 15397 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:48.192420 15397 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:48.193997 14787 ts_manager.cc:194] Registered new tserver with Master: 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:48.194763 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:45039
WARNING: no policy specified for oryx/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
I20260811 12:09:48.224910 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:48.542225 15405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:48.542491 15405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:48.542623 15405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:48.549554 15405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:48.549664 15405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:48.549809 15405 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:48.559104 15405 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:33371
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:48.560330 15405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:48.561714 15405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:48.573843 15417 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:48.573868 15422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:48.573805 15419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:48.574478 15405 server_base.cc:1034] running on GCE node
I20260811 12:09:48.575029 15405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:48.575933 15405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:48.577183 15405 hybrid_clock.cc:648] HybridClock initialized: now 1786450188577125 us; error 52 us; skew 500 ppm
I20260811 12:09:48.577482 15405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:48 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450188, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:48.580457 15405 init.cc:377] Logged in from keytab as oryx/127.4.48.195@KRBTEST.COM (short username oryx)
I20260811 12:09:48.581753 15405 webserver.cc:460] Webserver started at http://127.4.48.195:35685/ using document root <none> and password file <none>
I20260811 12:09:48.582391 15405 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:48.582500 15405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:48.582785 15405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:48.585578 15405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "40e2907186cd4c979b7d12122180a8c7"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "7415f22c20265e44affb4dd3504bab86"
server_key_iv: "e027235c578a525894189be340596896"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.586295 15405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "40e2907186cd4c979b7d12122180a8c7"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "7415f22c20265e44affb4dd3504bab86"
server_key_iv: "e027235c578a525894189be340596896"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.591233 15405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:09:48.594470 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.595726 15405 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 12:09:48.595872 15405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "40e2907186cd4c979b7d12122180a8c7"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "7415f22c20265e44affb4dd3504bab86"
server_key_iv: "e027235c578a525894189be340596896"
server_key_version: "encryptionkey@0"
I20260811 12:09:48.596031 15405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:48.623631 15405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:48.627547 15405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:48.627827 15405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:48.628784 15405 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:48.630234 15405 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:48.630311 15405 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.630424 15405 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:48.630472 15405 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:48.660879 15405 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44855
I20260811 12:09:48.660926 15631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44855 every 8 connection(s)
I20260811 12:09:48.662070 15405 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:48.663599 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 15405
I20260811 12:09:48.663784 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:09:48.664242 4291 external_mini_cluster.cc:1428] Setting key 5e3fd8060a0c746e85d167f97a6181ac
Aug 11 12:09:48 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450188, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:48.675271 15136 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
I20260811 12:09:48.681774 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:48.664637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:47301 (local address 127.4.48.254:33371)
0811 12:09:48.664872 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:48.664879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:48.666060 (+ 1181us) server_negotiation.cc:407] Connection header received
0811 12:09:48.668262 (+ 2202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:48.668266 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:48.668346 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:48.668443 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:48.669890 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.670622 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.672175 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.672568 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.672679 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:48.675934 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:48.675941 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:48.675946 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:48.675988 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:48.678696 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:48.679458 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:48.679463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:48.679466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:48.679620 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:48.680091 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:48.680096 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:48.680100 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:48.680386 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:48.680527 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:48.681214 (+ 687us) server_negotiation.cc:299] Negotiation successful
0811 12:09:48.681452 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 12:09:48.682672 15634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:48.665080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:47301)
0811 12:09:48.665911 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:48.665956 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:48.668017 (+ 2061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:48.668606 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:48.668614 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:48.669093 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:48.669706 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:48.669724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.670763 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.670767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:48.672040 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:48.672053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:48.672783 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:48.672791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:48.673008 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:48.673565 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:48.673587 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:48.675755 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:48.678827 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:48.678834 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:48.678848 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:48.679332 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:48.679763 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:48.679768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:48.679772 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:48.679970 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:48.680512 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:48.680520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:48.680849 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 12:09:48.681076 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:09:48.681578 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":421,"threads_started":1}
I20260811 12:09:48.683910 15632 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:48.684202 15632 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:48.684890 15632 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:48.686174 14787 ts_manager.cc:194] Registered new tserver with Master: 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855)
I20260811 12:09:48.686926 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:47301
I20260811 12:09:48.692320 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.4.48.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.253.keytab.
Entry for principal oryx/127.4.48.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.253.keytab.
I20260811 12:09:48.744141 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:37519
--webserver_interface=127.4.48.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.253.keytab
--principal=oryx/127.4.48.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:33371,127.4.48.253:37519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:49.061980 15637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:49.062263 15637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:49.062400 15637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:49.069430 15637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:49.069554 15637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:49.069628 15637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:49.069703 15637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:49.069761 15637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:49.079275 15637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:33371,127.4.48.253:37519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.253.keytab
--principal=oryx/127.4.48.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:37519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.48.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
W20260811 12:09:49.079620 15637 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.4.48.254:33371,127.4.48.253:37519'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 12:09:49.080641 15637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:49.081949 15637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:49.093072 15646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:49.093007 15647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:49.093050 15649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:49.094079 15637 server_base.cc:1034] running on GCE node
I20260811 12:09:49.094771 15637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:49.096091 15637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:49.097301 15637 hybrid_clock.cc:648] HybridClock initialized: now 1786450189097236 us; error 40 us; skew 500 ppm
I20260811 12:09:49.097597 15637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:49.100663 15637 init.cc:377] Logged in from keytab as oryx/127.4.48.253@KRBTEST.COM (short username oryx)
I20260811 12:09:49.102043 15637 webserver.cc:460] Webserver started at http://127.4.48.253:34395/ using document root <none> and password file <none>
I20260811 12:09:49.102684 15637 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:49.102782 15637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:49.103080 15637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:49.105896 15637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/instance:
uuid: "54629b4f2e5849189c73a42cf79c1a02"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "81b23b0a9c216224c13dadeb3649f716"
server_key_iv: "84b58f6f5987e15156c21f9e731bc34c"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.106628 15637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal/instance:
uuid: "54629b4f2e5849189c73a42cf79c1a02"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "81b23b0a9c216224c13dadeb3649f716"
server_key_iv: "84b58f6f5987e15156c21f9e731bc34c"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.111582 15637 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:09:49.114949 15661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:49.116308 15637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:49.116478 15637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "54629b4f2e5849189c73a42cf79c1a02"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "81b23b0a9c216224c13dadeb3649f716"
server_key_iv: "84b58f6f5987e15156c21f9e731bc34c"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.116664 15637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:49.128661 15637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:49.132242 15637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:49.132490 15637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:49.149398 15637 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.253:37519
I20260811 12:09:49.149433 15752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.253:37519 every 8 connection(s)
I20260811 12:09:49.150746 15637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
I20260811 12:09:49.152962 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 15637
I20260811 12:09:49.154596 15753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:49.163309 15753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.253@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:49.176807 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.155051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34762 (local address 127.4.48.253:37519)
0811 12:09:49.155796 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.155808 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.155849 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:09:49.157370 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.157391 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.157751 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.158072 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.159870 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.161158 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.162852 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.163510 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.163739 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.169503 (+ 5764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.169516 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.169529 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.169574 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.173056 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.173792 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.173800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.173809 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.173998 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.174397 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.174400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.174403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.174852 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.175139 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.175557 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.175948 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":314,"threads_started":1}
I20260811 12:09:49.180222 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.165942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.4.48.254:33371)
0811 12:09:49.166107 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.166113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.166711 (+ 598us) server_negotiation.cc:407] Connection header received
0811 12:09:49.166924 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.166929 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.167026 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.167097 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.168445 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.169160 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.170804 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.171221 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.171559 (+ 338us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.174812 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.174817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.174820 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.174851 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.177528 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.178184 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.178189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.178192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.178340 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.178731 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.178736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.178740 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.179032 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.179161 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.179644 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.179876 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:09:49.180253 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.165926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60814)
0811 12:09:49.166582 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.166615 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.166755 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.167348 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.167356 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.168033 (+ 677us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.168268 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.168277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.169345 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.169350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.170668 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.170680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.171622 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.171629 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.171819 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.172327 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.172348 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.174641 (+ 2293us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.177645 (+ 3004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.177656 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.177663 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.178075 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.178474 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.178477 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.178479 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.178627 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.179155 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.179162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.179348 (+ 186us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.179545 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.179877 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":350,"threads_started":1}
I20260811 12:09:49.181855 15753 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.253@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
I20260811 12:09:49.192325 15758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.182540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34766)
0811 12:09:49.182688 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.182702 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.182770 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.182986 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.182989 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.183265 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.183379 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.183387 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.184311 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.184314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.185124 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.185135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.185738 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.185742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.185859 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.186301 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.186317 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.188105 (+ 1788us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.190434 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.190438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.190440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.190785 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.191083 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.191086 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.191089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.191204 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.191630 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.191633 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.191796 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.191909 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.192109 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 12:09:49.192534 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.182595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34766 (local address 127.4.48.253:37519)
0811 12:09:49.182740 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.182744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.182758 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:09:49.182843 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.182847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.182896 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.182965 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.183493 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.184197 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.185241 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.185625 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.185709 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.188256 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.188261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.188264 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.188289 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.190317 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.190877 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.190880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.190882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.190990 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.191292 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.191295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.191297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.191541 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.191640 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.191991 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.192217 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":58}
I20260811 12:09:49.196862 15753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Bootstrap starting.
I20260811 12:09:49.197769 15397 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
I20260811 12:09:49.199892 15753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:49.200910 15753 log.cc:824] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:49.203267 15753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: No bootstrap required, opened a new log
I20260811 12:09:49.210042 15753 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } }
I20260811 12:09:49.210325 15753 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:49.210429 15753 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54629b4f2e5849189c73a42cf79c1a02, State: Initialized, Role: FOLLOWER
I20260811 12:09:49.210956 15753 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } }
I20260811 12:09:49.211762 15762 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } } }
I20260811 12:09:49.212060 15762 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:49.212697 15753 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:09:49.220722 15777 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:09:49.220870 15777 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:09:49.224251 15637 master_runner.cc:428] Detected that this master 54629b4f2e5849189c73a42cf79c1a02 is joining an existing cluster
I20260811 12:09:49.224337 15637 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.48.253:37519
I20260811 12:09:49.244220 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.227785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60818 (local address 127.4.48.254:33371)
0811 12:09:49.227962 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.227967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.228798 (+ 831us) server_negotiation.cc:407] Connection header received
0811 12:09:49.229096 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.229101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.229180 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.229238 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.230827 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.231983 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.234097 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.234750 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.234844 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.237520 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.237526 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.237530 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.237566 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.241117 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242022 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.242028 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.242031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.242193 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242634 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.242639 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.242642 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.242986 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.243094 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.243711 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.243910 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:09:49.244244 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.227959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34782 (local address 127.4.48.253:37519)
0811 12:09:49.228120 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.228128 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.229335 (+ 1207us) server_negotiation.cc:407] Connection header received
0811 12:09:49.229494 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.229498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.229565 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.229663 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.230970 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.231997 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.234081 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.234753 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.234844 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.237504 (+ 2660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.237510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.237514 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.237546 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.241105 (+ 3559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242038 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.242043 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.242045 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.242193 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242619 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.242623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.242626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.242973 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.243110 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.243635 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.243875 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:09:49.245199 15793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.228469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34782)
0811 12:09:49.229227 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.229244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.229380 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.229818 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.229822 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.230455 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.230841 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.230852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.232183 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.232187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.233939 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.233954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.235128 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.235134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.235347 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.236062 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.236088 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.237319 (+ 1231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.241238 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.241243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.241246 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.241945 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.242321 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242326 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.242332 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.242509 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.243187 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.243192 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.243366 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.244481 (+ 1115us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.244739 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":84,"spinlock_wait_cycles":34432,"thread_start_us":407,"threads_started":1}
I20260811 12:09:49.245759 15788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.227857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60818)
0811 12:09:49.228690 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.228710 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.228928 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.229438 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.229453 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.230202 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.230630 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.230643 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.232152 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.232157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.233939 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.233954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.235171 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.235179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.235375 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.236149 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.236166 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.237321 (+ 1155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.241257 (+ 3936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.241261 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.241264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.241887 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.242321 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.242330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.242333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.242518 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.243188 (+ 670us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.243192 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.243414 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.244872 (+ 1458us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.245176 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":65,"spinlock_wait_cycles":39808,"thread_start_us":478,"threads_started":1}
WARNING: no policy specified for oryx/127.4.48.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.252@KRBTEST.COM" created.
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.254@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
I20260811 12:09:49.271063 15757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.251562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.4.48.253:37519)
0811 12:09:49.251741 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.251747 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.256050 (+ 4303us) server_negotiation.cc:407] Connection header received
0811 12:09:49.256386 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.256390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.256458 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.256540 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.257650 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.258576 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.259903 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.260411 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.260524 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.264410 (+ 3886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.264417 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.264422 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.264467 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.267977 (+ 3510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.268804 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.268810 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.268814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.268973 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.269396 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.269401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.269404 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.269809 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.269965 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.270472 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.270714 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:09:49.271078 15795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.251557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34796)
0811 12:09:49.255916 (+ 4359us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.255954 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.256179 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.256617 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.256625 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.257280 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.257504 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.257513 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.258700 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.258706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.259756 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.259769 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.260571 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.260577 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.260765 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.261390 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.261413 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.264210 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.268127 (+ 3917us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.268140 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.268148 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.268669 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.269090 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.269094 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.269097 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.269257 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.269946 (+ 689us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.269955 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.270139 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.270328 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.270648 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4124,"thread_start_us":3960,"threads_started":1}
I20260811 12:09:49.273600 14787 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.48.253:37519: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02"
member_type: NON_VOTER
last_known_addr {
host: "127.4.48.253"
port: 37519
}
attrs {
promote: true
}
}
dest_uuid: "6c85c904d2514636985e638d1f662f20"
cas_config_opid_index: -1
I20260811 12:09:49.274871 14787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } }
I20260811 12:09:49.292102 15801 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } }
I20260811 12:09:49.294225 15714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:49.298231 15801 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.48.253:37519
I20260811 12:09:49.298430 15714 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.252.keytab.
Entry for principal oryx/127.4.48.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.252.keytab.
I20260811 12:09:49.299464 15637 master.cc:537] Master@127.4.48.253:37519 shutting down...
I20260811 12:09:49.300299 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:45923
--webserver_interface=127.4.48.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.252.keytab
--principal=oryx/127.4.48.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:09:49.300635 15806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } }
I20260811 12:09:49.300966 15806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:49.305994 15806 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:09:49.309719 14762 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 12:09:49.312187 14762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 -> Peer 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253:37519): Couldn't send request to peer 54629b4f2e5849189c73a42cf79c1a02. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:09:49.313457 15637 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:49.313913 15637 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:49.314100 15637 tablet_replica.cc:330] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: stopping tablet replica
I20260811 12:09:49.321972 15637 master.cc:552] Master@127.4.48.253:37519 shutdown complete.
I20260811 12:09:49.322090 15637 master_runner.cc:305] Clearing existing system tablet
I20260811 12:09:49.322618 15637 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:49.322718 15637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:49.325824 15637 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 12:09:49.327504 15815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:49.327965 15637 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:49.328377 15637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:09:49.328524 15637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "54629b4f2e5849189c73a42cf79c1a02"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "81b23b0a9c216224c13dadeb3649f716"
server_key_iv: "84b58f6f5987e15156c21f9e731bc34c"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.333964 15637 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:49.340812 15637 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:09:49.341045 15637 log.cc:1196] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 12:09:49.341529 15637 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Deleting consensus metadata
I20260811 12:09:49.341836 15637 master_runner.cc:315] Copying system tablet from 127.4.48.254:33371
I20260811 12:09:49.344622 15637 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: tablet copy: Beginning tablet copy session from remote peer at address 127.4.48.254:33371
I20260811 12:09:49.367141 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.345827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60834 (local address 127.4.48.254:33371)
0811 12:09:49.346004 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.346010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.352353 (+ 6343us) server_negotiation.cc:407] Connection header received
0811 12:09:49.352560 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.352566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.352656 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.352772 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.353805 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.354993 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.356931 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.357562 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.357688 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.360152 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.360159 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.360164 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.360207 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.364011 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.364780 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.364785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.364789 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.364950 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.365416 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.365421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.365425 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.365790 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.365954 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.366531 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.366770 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:09:49.367398 15829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.345771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60834)
0811 12:09:49.352215 (+ 6444us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.352236 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.352424 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.352816 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.352821 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.353358 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.353647 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.353659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.355143 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.355149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.356748 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.356769 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.357734 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.357740 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.357949 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.358680 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.358702 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.359921 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.364146 (+ 4225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.364151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.364155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.364648 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.365082 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.365087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.365090 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.365296 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.365920 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.365924 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.366121 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.366348 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.366727 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6285,"thread_start_us":6142,"threads_started":1}
I20260811 12:09:49.368633 14827 tablet_copy_service.cc:140] P 6c85c904d2514636985e638d1f662f20: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 54629b4f2e5849189c73a42cf79c1a02 ({username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:60834)
I20260811 12:09:49.369148 14827 tablet_copy_service.cc:161] P 6c85c904d2514636985e638d1f662f20: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 54629b4f2e5849189c73a42cf79c1a02 at {username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:60834: session id = 54629b4f2e5849189c73a42cf79c1a02-00000000000000000000000000000000
I20260811 12:09:49.374588 14827 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:09:49.377543 15637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:09:49.384979 15637 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: tablet copy: Starting download of 0 data blocks...
I20260811 12:09:49.385525 15637 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: tablet copy: Starting download of 1 WAL segments...
I20260811 12:09:49.389727 15637 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:09:49.394639 14827 tablet_copy_service.cc:342] P 6c85c904d2514636985e638d1f662f20: Request end of tablet copy session 54629b4f2e5849189c73a42cf79c1a02-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:60834
I20260811 12:09:49.395006 14827 tablet_copy_service.cc:433] P 6c85c904d2514636985e638d1f662f20: ending tablet copy session 54629b4f2e5849189c73a42cf79c1a02-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 54629b4f2e5849189c73a42cf79c1a02
I20260811 12:09:49.408965 15637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:49.411659 15832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:49.412113 15833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:49.423946 15637 server_base.cc:1034] running on GCE node
W20260811 12:09:49.425007 15839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:49.425504 15637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:49.426210 15637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:49.427453 15637 hybrid_clock.cc:648] HybridClock initialized: now 1786450189427373 us; error 33 us; skew 500 ppm
I20260811 12:09:49.427740 15637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:49.430084 15637 init.cc:377] Logged in from keytab as oryx/127.4.48.253@KRBTEST.COM (short username oryx)
I20260811 12:09:49.431579 15637 webserver.cc:460] Webserver started at http://127.4.48.253:37945/ using document root <none> and password file <none>
I20260811 12:09:49.432008 15637 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:49.432107 15637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:49.435297 15637 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:09:49.437464 15849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:49.437906 15637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 12:09:49.438061 15637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "54629b4f2e5849189c73a42cf79c1a02"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "81b23b0a9c216224c13dadeb3649f716"
server_key_iv: "84b58f6f5987e15156c21f9e731bc34c"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.438215 15637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:49.502756 15637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:49.507162 15637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:49.551839 15637 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.253:37519
I20260811 12:09:49.551891 15949 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.253:37519 every 8 connection(s)
I20260811 12:09:49.552814 15637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
I20260811 12:09:49.559652 15952 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:09:49.564990 15952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Bootstrap starting.
I20260811 12:09:49.588145 15952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:09:49.588866 15952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Bootstrap complete.
I20260811 12:09:49.590121 15952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } }
I20260811 12:09:49.590447 15952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 54629b4f2e5849189c73a42cf79c1a02, State: Initialized, Role: LEARNER
I20260811 12:09:49.590771 15952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } }
I20260811 12:09:49.592242 15952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:49.591995 15957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } }
I20260811 12:09:49.592567 15957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:09:49.604518 15978 catalog_manager.cc:1259] Loaded cluster ID: f90e8db32f08402080f23baa79fe4c52
I20260811 12:09:49.604619 15978 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: loading cluster ID for follower catalog manager: success
I20260811 12:09:49.608273 15978 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: acquiring CA information for follower catalog manager: success
I20260811 12:09:49.611064 15978 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:09:49.689332 15632 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
W20260811 12:09:49.744477 15810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:49.744755 15810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:49.744867 15810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:49.752091 15810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:49.752197 15810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:49.752259 15810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:49.752324 15810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:49.752377 15810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:49.761610 15810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.252.keytab
--principal=oryx/127.4.48.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:45923
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.48.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:49.762732 15810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:49.763960 15810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:49.775203 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:49.775360 15985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:49.775283 15983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:49.776254 15810 server_base.cc:1034] running on GCE node
I20260811 12:09:49.776981 15810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:49.778252 15810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:49.779475 15810 hybrid_clock.cc:648] HybridClock initialized: now 1786450189779401 us; error 47 us; skew 500 ppm
I20260811 12:09:49.779862 15810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:49.782913 15810 init.cc:377] Logged in from keytab as oryx/127.4.48.252@KRBTEST.COM (short username oryx)
I20260811 12:09:49.784291 15810 webserver.cc:460] Webserver started at http://127.4.48.252:40325/ using document root <none> and password file <none>
I20260811 12:09:49.784930 15810 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:49.785064 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:49.785351 15810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:49.788411 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/instance:
uuid: "cfc348bd49744de08a3b76aecea07d98"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "413fcaff20683f094c5d3c775af6f2ff"
server_key_iv: "7260ca4b88bc5128409eae9e644cf657"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.789194 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal/instance:
uuid: "cfc348bd49744de08a3b76aecea07d98"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "413fcaff20683f094c5d3c775af6f2ff"
server_key_iv: "7260ca4b88bc5128409eae9e644cf657"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.794495 15810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:09:49.798054 16002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:49.799535 15810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:49.799774 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "cfc348bd49744de08a3b76aecea07d98"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "413fcaff20683f094c5d3c775af6f2ff"
server_key_iv: "7260ca4b88bc5128409eae9e644cf657"
server_key_version: "encryptionkey@0"
I20260811 12:09:49.800093 15810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:49.814551 15810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:49.818418 15810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:49.818661 15810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:49.838806 15810 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.252:45923
I20260811 12:09:49.838943 16096 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.252:45923 every 8 connection(s)
I20260811 12:09:49.840214 15810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
I20260811 12:09:49.842592 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 15810
I20260811 12:09:49.844725 16100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:49.852386 16100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.252@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:49.869333 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.845017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37304 (local address 127.4.48.252:45923)
0811 12:09:49.845838 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.845850 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.845901 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:09:49.847537 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.847559 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.848003 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.848348 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.849933 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.851324 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.852613 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.853337 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.853498 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.858472 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.858482 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.858495 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.858547 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.863384 (+ 4837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.864351 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.864361 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.864371 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.864600 (+ 229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.865105 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.865110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.865114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.865794 (+ 680us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.866119 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.867420 (+ 1301us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.867885 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":389,"threads_started":1}
I20260811 12:09:49.872588 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.854817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60850)
0811 12:09:49.855573 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.855606 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.855804 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.856168 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.856174 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.856845 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.857135 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.857143 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.858149 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.858153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.859831 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.859851 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.860795 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.860802 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.860998 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.861622 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.861642 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.864981 (+ 3339us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.869527 (+ 4546us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.869539 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.869545 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.870170 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.870638 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.870643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.870646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.870829 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.871503 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.871510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.871730 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.871950 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.872249 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":420,"threads_started":1}
I20260811 12:09:49.872999 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.854868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.4.48.254:33371)
0811 12:09:49.855014 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.855018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.855728 (+ 710us) server_negotiation.cc:407] Connection header received
0811 12:09:49.855928 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.855932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.856011 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.856095 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.857280 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.858000 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.859976 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.860610 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.860729 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.865229 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.865236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.865242 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.865285 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.869339 (+ 4054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.870325 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.870331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.870334 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.870486 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.870967 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.870973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.870977 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.871365 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.871515 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.872164 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.872382 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:09:49.874128 16100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:09:49.877807 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.861914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34808)
0811 12:09:49.870695 (+ 8781us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.870710 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.870801 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.872544 (+ 1743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.872549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.872578 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:49.872801 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.872811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.874377 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.874381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.875936 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.875949 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.877093 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.877099 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.877263 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.877324 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.877407 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.877529 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8670,"thread_start_us":8528,"threads_started":1}
I20260811 12:09:49.877911 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.862111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34808 (local address 127.4.48.253:37519)
0811 12:09:49.872163 (+ 10052us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.872169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.872188 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:09:49.872253 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.872258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.872361 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.872485 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:49.872950 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.874212 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.876106 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.876919 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.877034 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.877247 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.877448 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.877579 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9911,"thread_start_us":9773,"threads_started":1}
I20260811 12:09:49.879348 15901 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 12:09:49.880051 15957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c85c904d2514636985e638d1f662f20. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } }
I20260811 12:09:49.880314 15957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: LEARNER
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.252@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
I20260811 12:09:49.889225 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.874894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.4.48.253:37519)
0811 12:09:49.878131 (+ 3237us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.878135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.878149 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:09:49.878183 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.878186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.878235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.878335 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.879188 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.880300 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.881525 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.881939 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.882026 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.884724 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.884728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.884731 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.884760 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.887221 (+ 2461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.887800 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.887804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.887806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.887937 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.888224 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.888227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.888229 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.888459 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.888561 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.888817 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.888971 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3129,"thread_start_us":3908,"threads_started":1}
I20260811 12:09:49.889292 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.874866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34810)
0811 12:09:49.875049 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.875066 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.875163 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.878364 (+ 3201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.878369 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.878839 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.879033 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.879044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.880427 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.880432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.881412 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.881425 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.882062 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.882065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.882212 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.882712 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.882728 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.884555 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.887328 (+ 2773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.887331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.887334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.887712 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.888015 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.888018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.888020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.888153 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.888537 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.888540 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.888653 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.888746 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.888970 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:09:49.890177 16100 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450189, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.252@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
I20260811 12:09:49.902185 16106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.890752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37308)
0811 12:09:49.890964 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.890978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.891050 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.891276 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.891278 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.891605 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.891816 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.891847 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.892787 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.892791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.893685 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.893697 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.894409 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.894413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.894584 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.895048 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.895063 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.897153 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.900104 (+ 2951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.900109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.900113 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.900497 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.900831 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.900835 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.900837 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.900994 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.901457 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.901461 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.901586 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.901697 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.901868 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:09:49.902267 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.890842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.4.48.252:45923)
0811 12:09:49.891027 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.891034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.891059 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:09:49.891116 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.891120 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.891175 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.891263 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.891951 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.892679 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.893806 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.894269 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.894368 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.897350 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.897355 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.897358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.897387 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.899983 (+ 2596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.900595 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.900599 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.900601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.900737 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.901085 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.901088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.901090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.901339 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.901442 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.901772 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.901995 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":74}
I20260811 12:09:49.906810 16100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Bootstrap starting.
I20260811 12:09:49.910107 16100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:49.911125 16100 log.cc:824] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:49.913676 16100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: No bootstrap required, opened a new log
I20260811 12:09:49.920874 16100 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } }
I20260811 12:09:49.921155 16100 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:49.921262 16100 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfc348bd49744de08a3b76aecea07d98, State: Initialized, Role: FOLLOWER
I20260811 12:09:49.921767 16100 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } }
I20260811 12:09:49.922695 16118 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } } }
I20260811 12:09:49.923120 16118 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:49.923789 16100 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:09:49.932591 16138 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:09:49.932713 16138 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:09:49.936157 15810 master_runner.cc:428] Detected that this master cfc348bd49744de08a3b76aecea07d98 is joining an existing cluster
I20260811 12:09:49.936259 15810 master_runner.cc:432] Initiating AddMaster RPC to add 127.4.48.252:45923
I20260811 12:09:49.957072 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.940681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.4.48.254:33371)
0811 12:09:49.940853 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.940859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.941181 (+ 322us) server_negotiation.cc:407] Connection header received
0811 12:09:49.941327 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.941332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.941407 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.941486 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.942663 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.943506 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.946251 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.946965 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.947086 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.949978 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.949986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.949991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.950029 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.953707 (+ 3678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.954486 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.954492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.954495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.954684 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.955198 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.955204 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.955207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.955578 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.955721 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.956425 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.956677 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:09:49.958145 16149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.940202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60860)
0811 12:09:49.941084 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.941099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.941219 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.941523 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.941528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.942132 (+ 604us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.942525 (+ 393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.942536 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.943629 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.943633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.946018 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.946036 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.947364 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.947370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.947562 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.948358 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.948380 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.949745 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.953880 (+ 4135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.953885 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.953888 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.954348 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.954857 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.954861 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.954865 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.955059 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.955796 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.955802 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.956112 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.957253 (+ 1141us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.957558 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":781,"thread_start_us":425,"threads_started":1}
I20260811 12:09:49.961360 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.940175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34814 (local address 127.4.48.253:37519)
0811 12:09:49.940337 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.940342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.944432 (+ 4090us) server_negotiation.cc:407] Connection header received
0811 12:09:49.944689 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.944695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.944767 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.944837 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.946831 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.947968 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.950591 (+ 2623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.951212 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.951310 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.953945 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.953952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.953957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.953995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.957455 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.958752 (+ 1297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.958757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.958760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.958959 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.959469 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.959474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.959477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.959896 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.960012 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.960753 (+ 741us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.960986 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:09:49.962574 16150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.940287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34814)
0811 12:09:49.944323 (+ 4036us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.944344 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.944539 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.945045 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.945049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.946356 (+ 1307us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.946690 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.946703 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.948464 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.948473 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.950428 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.950443 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.951625 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.951631 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.951844 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.952514 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.952540 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.953709 (+ 1169us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.957865 (+ 4156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.957870 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.957873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.958607 (+ 734us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.959113 (+ 506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.959118 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.959122 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.959319 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.960106 (+ 787us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.960111 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.960358 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.961764 (+ 1406us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.962088 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3898,"mutex_wait_us":1,"spinlock_wait_cycles":22272,"thread_start_us":3703,"threads_started":1}
I20260811 12:09:49.964957 16152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.944522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37324)
0811 12:09:49.947095 (+ 2573us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.947112 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.947245 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.947702 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.947705 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.948496 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.949093 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.949108 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.950613 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.950621 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.952368 (+ 1747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.952383 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.953764 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.953771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.953966 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.954551 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.954802 (+ 251us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.956234 (+ 1432us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:49.961047 (+ 4813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.961052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.961055 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.961589 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.962062 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:49.962065 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:49.962068 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:49.962217 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:49.963116 (+ 899us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:49.963119 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:49.964074 (+ 955us) client_negotiation.cc:769] Sending connection context
0811 12:09:49.964207 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 12:09:49.964490 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2435,"thread_start_us":1372,"threads_started":1}
I20260811 12:09:49.965230 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.944722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.4.48.252:45923)
0811 12:09:49.944916 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.944921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.947172 (+ 2251us) server_negotiation.cc:407] Connection header received
0811 12:09:49.947374 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.947379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.947447 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.947508 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.949252 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.950420 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.952593 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.953298 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.953419 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.956435 (+ 3016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.956442 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.956446 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.956480 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.960484 (+ 4004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.961723 (+ 1239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.961728 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.961731 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.961920 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.962341 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.962346 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.962349 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.962869 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.963051 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.964338 (+ 1287us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.964627 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":59}
I20260811 12:09:49.975322 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 14893
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.193@KRBTEST.COM
Aug 11 12:09:49 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.254@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
I20260811 12:09:49.996134 16155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.976384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60750 (local address 127.4.48.193:35003)
0811 12:09:49.977001 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.977007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.977032 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:09:49.977099 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.977105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.977285 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.977433 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.979161 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.980386 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.982259 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.983039 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.983773 (+ 734us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.988888 (+ 5115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.988904 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.988922 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.988981 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:49.993149 (+ 4168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.993895 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.993902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.993905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.994035 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:49.994406 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:49.994410 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:49.994412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:49.994819 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:49.995149 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:49.995478 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 12:09:49.995814 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":308,"threads_started":1}
I20260811 12:09:49.997731 15124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:50.003260 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.963563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37336)
0811 12:09:49.963853 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:49.963872 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:49.963964 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:49.965950 (+ 1986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:49.965955 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:49.966281 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:49.966430 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.966437 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.988738 (+ 22301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.988744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.989748 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:49.989765 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.990608 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.990613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:49.990776 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.991356 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:49.991374 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:49.993587 (+ 2213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:50.000302 (+ 6715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.000307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.000311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.000855 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.001385 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.001391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.001395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.001554 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.002226 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:50.002232 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:50.002445 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.002569 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.002874 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260811 12:09:50.004482 16104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:49.963872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.4.48.252:45923)
0811 12:09:49.965616 (+ 1744us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:49.965621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:49.965637 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:49.965724 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:49.965727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:49.965806 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:49.965894 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:49.987882 (+ 21988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.988596 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.989975 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:49.990465 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:49.990571 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:49.993755 (+ 3184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:49.993762 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:49.993766 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:49.993802 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.000140 (+ 6338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.001088 (+ 948us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.001093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.001097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.001251 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.001696 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.001701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.001705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.002083 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.003626 (+ 1543us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.003824 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.004081 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1646,"thread_start_us":1516,"threads_started":1}
I20260811 12:09:50.006012 14787 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.4.48.252:45923: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "cfc348bd49744de08a3b76aecea07d98"
member_type: NON_VOTER
last_known_addr {
host: "127.4.48.252"
port: 45923
}
attrs {
promote: true
}
}
dest_uuid: "6c85c904d2514636985e638d1f662f20"
cas_config_opid_index: 5
I20260811 12:09:50.007123 14787 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } }
I20260811 12:09:50.008924 16107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cfc348bd49744de08a3b76aecea07d98 (127.4.48.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.010663 15901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:09:50.012501 16050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:50.015036 16157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260811 12:09:50.018151 16050 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 12:09:50.020946 16107 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.4.48.252:45923
I20260811 12:09:50.021556 15810 master.cc:537] Master@127.4.48.252:45923 shutting down...
I20260811 12:09:50.021143 16158 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260811 12:09:50.021610 16157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.022732 16157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
W20260811 12:09:50.034178 14762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 -> Peer cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923): Couldn't send request to peer cfc348bd49744de08a3b76aecea07d98. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 12:09:50.036521 15810 raft_consensus.cc:2229] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:50.036907 15810 raft_consensus.cc:2258] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:50.037104 15810 tablet_replica.cc:330] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: stopping tablet replica
I20260811 12:09:50.039520 15901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER cfc348bd49744de08a3b76aecea07d98 (127.4.48.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.044205 15810 master.cc:552] Master@127.4.48.252:45923 shutdown complete.
I20260811 12:09:50.044327 15810 master_runner.cc:305] Clearing existing system tablet
I20260811 12:09:50.044747 15810 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:50.044909 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:50.046020 15957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: NON_VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: true } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.046303 15957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:09:50.046833 16157 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: attempting to promote NON_VOTER 54629b4f2e5849189c73a42cf79c1a02 to VOTER
I20260811 12:09:50.047787 16157 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } }
I20260811 12:09:50.049666 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.049159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37346)
0811 12:09:50.049324 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.049459 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:09:50.050980 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.050576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37362)
0811 12:09:50.050762 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.050851 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:45923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 12:09:50.051221 14762 consensus_peers.cc:487] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 -> Peer cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923): Couldn't send request to peer cfc348bd49744de08a3b76aecea07d98. Status: Network error: Client connection negotiation failed: client connection to 127.4.48.252:45923: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:09:50.051702 15901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 LEARNER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 12:09:50.052258 16157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 12:09:50.054440 16158 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.055166 15901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.058480 16107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.058714 16107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:50.058826 15957 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.059141 15957 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:50.059262 16107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.059485 16107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:50.064450 15810 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.007s sys 0.003s
I20260811 12:09:50.066373 16167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:50.066747 15810 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:50.067037 15810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:09:50.067167 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "cfc348bd49744de08a3b76aecea07d98"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "413fcaff20683f094c5d3c775af6f2ff"
server_key_iv: "7260ca4b88bc5128409eae9e644cf657"
server_key_version: "encryptionkey@0"
I20260811 12:09:50.072484 15810 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:50.080081 15810 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 12:09:50.080258 15810 log.cc:1196] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 12:09:50.080778 15810 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Deleting consensus metadata
I20260811 12:09:50.081043 15810 master_runner.cc:315] Copying system tablet from 127.4.48.254:33371
I20260811 12:09:50.096303 15810 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: tablet copy: Beginning tablet copy session from remote peer at address 127.4.48.254:33371
I20260811 12:09:50.113730 15400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.097243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60870 (local address 127.4.48.254:33371)
0811 12:09:50.097500 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.097508 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.098443 (+ 935us) server_negotiation.cc:407] Connection header received
0811 12:09:50.098705 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.098711 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.098808 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.098908 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:50.099992 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.101174 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.103126 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.103829 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.103939 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.106462 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:50.106470 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:50.106474 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:50.106518 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.110641 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.111389 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.111394 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.111398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.111590 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.112014 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.112019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.112022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.112390 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.112543 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.113027 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.113267 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 12:09:50.113757 16184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.097591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60870)
0811 12:09:50.098315 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.098346 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:50.098516 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:50.098963 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:50.098969 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:50.099522 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:50.099846 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.099860 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.101338 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.101344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.102963 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.102978 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.104144 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.104150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.104327 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.105059 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:50.105081 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:50.106183 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:50.110769 (+ 4586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.110774 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.110779 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.111274 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.111733 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.111738 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.111742 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.111906 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.112527 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:50.112532 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:50.112752 (+ 220us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.112899 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.113246 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":360,"threads_started":1}
I20260811 12:09:50.114665 14827 tablet_copy_service.cc:140] P 6c85c904d2514636985e638d1f662f20: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer cfc348bd49744de08a3b76aecea07d98 ({username='oryx', principal='oryx/127.4.48.252@KRBTEST.COM'} at 127.0.0.1:60870)
I20260811 12:09:50.114848 14827 tablet_copy_service.cc:161] P 6c85c904d2514636985e638d1f662f20: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer cfc348bd49744de08a3b76aecea07d98 at {username='oryx', principal='oryx/127.4.48.252@KRBTEST.COM'} at 127.0.0.1:60870: session id = cfc348bd49744de08a3b76aecea07d98-00000000000000000000000000000000
I20260811 12:09:50.118700 14827 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 12:09:50.120653 15810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 12:09:50.127874 15810 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: tablet copy: Starting download of 0 data blocks...
I20260811 12:09:50.128156 15810 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: tablet copy: Starting download of 1 WAL segments...
I20260811 12:09:50.129335 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 14893
I20260811 12:09:50.140940 15810 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 12:09:50.145599 14827 tablet_copy_service.cc:342] P 6c85c904d2514636985e638d1f662f20: Request end of tablet copy session cfc348bd49744de08a3b76aecea07d98-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.4.48.252@KRBTEST.COM'} at 127.0.0.1:60870
I20260811 12:09:50.145758 14827 tablet_copy_service.cc:433] P 6c85c904d2514636985e638d1f662f20: ending tablet copy session cfc348bd49744de08a3b76aecea07d98-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer cfc348bd49744de08a3b76aecea07d98
I20260811 12:09:50.157308 15810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:50.159869 16185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:50.161141 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:35003
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:33371
--webserver_port=34973
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:50.165802 16186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:50.177222 16189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:50.177651 15810 server_base.cc:1034] running on GCE node
I20260811 12:09:50.177942 15810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:50.178548 15810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:50.179751 15810 hybrid_clock.cc:648] HybridClock initialized: now 1786450190179706 us; error 34 us; skew 500 ppm
I20260811 12:09:50.179968 15810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450190, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:50.182212 15810 init.cc:377] Logged in from keytab as oryx/127.4.48.252@KRBTEST.COM (short username oryx)
I20260811 12:09:50.184376 15810 webserver.cc:460] Webserver started at http://127.4.48.252:32965/ using document root <none> and password file <none>
I20260811 12:09:50.184696 15810 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:50.184803 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:50.187856 15810 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:09:50.189805 16203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:50.190608 15810 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 12:09:50.190732 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "cfc348bd49744de08a3b76aecea07d98"
format_stamp: "Formatted at 2026-08-11 12:09:49 on dist-test-slave-nj21"
server_key: "413fcaff20683f094c5d3c775af6f2ff"
server_key_iv: "7260ca4b88bc5128409eae9e644cf657"
server_key_version: "encryptionkey@0"
I20260811 12:09:50.190917 15810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:50.206543 15810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:50.210459 15810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:50.234440 15810 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.252:45923
I20260811 12:09:50.234495 16290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.252:45923 every 8 connection(s)
I20260811 12:09:50.235409 15810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
I20260811 12:09:50.240617 16293 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:09:50.244460 16293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Bootstrap starting.
I20260811 12:09:50.263454 16293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:09:50.264010 16293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: Bootstrap complete.
I20260811 12:09:50.265182 16293 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } }
I20260811 12:09:50.265509 16293 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: cfc348bd49744de08a3b76aecea07d98, State: Initialized, Role: LEARNER
I20260811 12:09:50.265803 16293 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } }
I20260811 12:09:50.266641 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.266942 16293 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:50.267073 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:09:50.278352 16322 catalog_manager.cc:1259] Loaded cluster ID: f90e8db32f08402080f23baa79fe4c52
I20260811 12:09:50.278473 16322 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: loading cluster ID for follower catalog manager: success
I20260811 12:09:50.281971 16322 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: acquiring CA information for follower catalog manager: success
I20260811 12:09:50.284349 16322 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:09:50.530018 16193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:50.530416 16193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:50.530586 16193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:50.537657 16193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:50.537773 16193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:50.537920 16193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:50.547343 16193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:35003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=34973
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:50.548650 16193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:50.549932 16193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:50.564127 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:50.564283 16330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:50.564678 16334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:50.564910 16193 server_base.cc:1034] running on GCE node
I20260811 12:09:50.565490 16193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:50.566334 16193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:50.567530 16193 hybrid_clock.cc:648] HybridClock initialized: now 1786450190567472 us; error 48 us; skew 500 ppm
I20260811 12:09:50.567849 16193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450190, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:50.571157 16193 init.cc:377] Logged in from keytab as oryx/127.4.48.193@KRBTEST.COM (short username oryx)
I20260811 12:09:50.572651 16193 webserver.cc:460] Webserver started at http://127.4.48.193:34973/ using document root <none> and password file <none>
I20260811 12:09:50.573395 16193 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:50.573537 16193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:50.578815 16193 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:09:50.581589 16347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:50.582901 16193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:50.583061 16193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "0f9652be36c94c80b06d8c2d1d62f91e"
format_stamp: "Formatted at 2026-08-11 12:09:47 on dist-test-slave-nj21"
server_key: "143233e9bfa2646d4ce788a2d6c8177e"
server_key_iv: "a274768dc8caa077f65ae7b8dff85413"
server_key_version: "encryptionkey@0"
I20260811 12:09:50.584100 16193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:50.606823 16193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:50.610962 16193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:50.611227 16193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:50.612267 16193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:50.613821 16193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:50.613931 16193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:50.614068 16193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:50.614147 16193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:50.645782 16193 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35003
I20260811 12:09:50.645852 16550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35003 every 8 connection(s)
I20260811 12:09:50.647264 16193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:50.654217 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 16193
I20260811 12:09:50.654584 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 15150
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450190, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450190, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450190, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
Aug 11 12:09:50 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.194@KRBTEST.COM
I20260811 12:09:50.701685 16562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.650182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:50611 (local address 127.4.48.252:45923)
0811 12:09:50.650932 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.650936 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.652022 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 12:09:50.654823 (+ 2801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.654830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.654919 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.655034 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:50.666344 (+ 11310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.667936 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.676309 (+ 8373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.676958 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.677057 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.693692 (+ 16635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:50.693699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:50.693704 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:50.693751 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.697839 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.699241 (+ 1402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.699246 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.699250 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.699397 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.699925 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.699930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.699932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.700262 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.700397 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.701100 (+ 703us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.701309 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":613,"thread_start_us":462,"threads_started":1}
I20260811 12:09:50.701867 16561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.650182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:40681 (local address 127.4.48.254:33371)
0811 12:09:50.650730 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.650737 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.651960 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 12:09:50.656085 (+ 4125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.656091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.656174 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.656264 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:50.668167 (+ 11903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.669306 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.678223 (+ 8917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.678892 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.678989 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.689886 (+ 10897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:50.689892 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:50.689897 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:50.689940 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.694876 (+ 4936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.698546 (+ 3670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.698552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.698555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.698743 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.699941 (+ 1198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.699946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.699949 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.700260 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.700417 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.701266 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.701514 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":225,"threads_started":1}
I20260811 12:09:50.702649 16569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.652476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56377 (local address 127.4.48.253:37519)
0811 12:09:50.653157 (+ 681us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.653164 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.653209 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:09:50.655540 (+ 2331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.655547 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.655648 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.655794 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:50.664483 (+ 8689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.665676 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.674268 (+ 8592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.674851 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.674984 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.688942 (+ 13958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:50.688950 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:50.688955 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:50.688997 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.696766 (+ 7769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.698240 (+ 1474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.698245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.698249 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.698402 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.698968 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.698973 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.698976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.699416 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.699562 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.702038 (+ 2476us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.702267 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":811,"threads_started":1}
I20260811 12:09:50.703637 16563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.650741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.4.48.193:50611)
0811 12:09:50.651762 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.651823 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:50.654386 (+ 2563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:50.659869 (+ 5483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:50.659874 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:50.665855 (+ 5981us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:50.666147 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.666160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.670744 (+ 4584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.670751 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.672569 (+ 1818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.672584 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.677445 (+ 4861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.677452 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.677751 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.680406 (+ 2655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:50.680438 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:50.688690 (+ 8252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:50.698570 (+ 9880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.698574 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.698577 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.699104 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.699557 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.699561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.699565 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.699777 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.700462 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:50.700467 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:50.700671 (+ 204us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.700813 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.702355 (+ 1542us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"mutex_wait_us":1226,"spinlock_wait_cycles":14336,"thread_start_us":401,"threads_started":1}
I20260811 12:09:50.703766 16565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.651126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:40681)
0811 12:09:50.651797 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.651817 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:50.655274 (+ 3457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:50.658743 (+ 3469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:50.658748 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:50.659343 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:50.659653 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.659665 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.669573 (+ 9908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.669579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.676106 (+ 6527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.676122 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.679279 (+ 3157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.679285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.679455 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.682629 (+ 3174us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:50.682653 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:50.687209 (+ 4556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:50.697982 (+ 10773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.697985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.697988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.698431 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.699328 (+ 897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.699333 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.699336 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.699822 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.700546 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:50.700550 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:50.700670 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.701957 (+ 1287us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.702662 (+ 705us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":442,"spinlock_wait_cycles":36992,"thread_start_us":347,"threads_started":1}
I20260811 12:09:50.704300 16568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.652357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.4.48.193:56377)
0811 12:09:50.653103 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.653125 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:50.655378 (+ 2253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:50.656726 (+ 1348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:50.656738 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:50.657621 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:50.658485 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.658510 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.672750 (+ 14240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.672755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.674019 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.674036 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.679819 (+ 5783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.679825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.679985 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.684049 (+ 4064us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:50.684069 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:50.687989 (+ 3920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:50.696926 (+ 8937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.696935 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.696952 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.697666 (+ 714us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.698533 (+ 867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:50.698538 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:50.698541 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:50.698791 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:50.699657 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:50.699665 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:50.700127 (+ 462us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.700544 (+ 417us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.702195 (+ 1651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":294,"threads_started":1}
I20260811 12:09:50.705261 16558 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:50.705456 16559 heartbeater.cc:347] Connected to a master server at 127.4.48.253:37519
I20260811 12:09:50.705485 16555 heartbeater.cc:347] Connected to a master server at 127.4.48.252:45923
I20260811 12:09:50.705621 16555 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:50.705626 16558 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:50.705629 16559 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:50.706646 16558 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:50.706671 16559 heartbeater.cc:511] Master 127.4.48.253:37519 requested a full tablet report, sending...
I20260811 12:09:50.707181 16555 heartbeater.cc:511] Master 127.4.48.252:45923 requested a full tablet report, sending...
I20260811 12:09:50.708312 16575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.656148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36710 (local address 127.4.48.194:41393)
0811 12:09:50.683058 (+ 26910us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.683067 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.683722 (+ 655us) server_negotiation.cc:407] Connection header received
0811 12:09:50.683803 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.683811 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.684182 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.684339 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:50.686726 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.688456 (+ 1730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.690397 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.691198 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.691315 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.698532 (+ 7217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:50.698546 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:50.698563 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:50.698607 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:50.702664 (+ 4057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.705088 (+ 2424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.705096 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.705099 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.705266 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:50.705726 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:50.705730 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:50.705734 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:50.706139 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:50.706433 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.707580 (+ 1147us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.707884 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26714,"thread_start_us":33991,"threads_started":1}
I20260811 12:09:50.708700 16571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.653715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37370 (local address 127.4.48.252:45923)
0811 12:09:50.696637 (+ 42922us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:50.696642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:50.697024 (+ 382us) server_negotiation.cc:407] Connection header received
0811 12:09:50.697159 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:50.697164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:50.697268 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:50.697386 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:50.698208 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.700994 (+ 2786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.704187 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.707349 (+ 3162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.707482 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.707723 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:50.708215 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 12:09:50.708371 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42783,"thread_start_us":42622,"threads_started":1}
I20260811 12:09:50.709296 16232 ts_manager.cc:194] Registered new tserver with Master: 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003)
I20260811 12:09:50.709313 15886 ts_manager.cc:194] Registered new tserver with Master: 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003)
I20260811 12:09:50.709389 16570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:50.653485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37370)
0811 12:09:50.696924 (+ 43439us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:50.696942 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:50.697051 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:50.697689 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:50.697694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:50.697723 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:50.698040 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.698053 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.701105 (+ 3052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.701111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.704022 (+ 2911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:50.704037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:50.707823 (+ 3786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:50.707829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:50.708003 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:50.708035 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:09:50.708855 (+ 820us) client_negotiation.cc:240] Negotiation successful
0811 12:09:50.708994 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43323,"thread_start_us":42696,"threads_started":1}
I20260811 12:09:50.709702 15387 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:50.710817 16246 raft_consensus.cc:1203] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 12:09:50.711011 14787 ts_manager.cc:194] Re-registered known tserver with Master: 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003)
I20260811 12:09:50.712199 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:40681
I20260811 12:09:50.712714 16302 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c85c904d2514636985e638d1f662f20. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:50.712989 16302 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: LEARNER
I20260811 12:09:50.795301 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 15150
I20260811 12:09:50.822273 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:41393
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:33371
--webserver_port=43661
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:51.151468 16581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:51.151779 16581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:51.151917 16581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:51.158875 16581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:51.158983 16581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:51.159135 16581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:51.168326 16581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:41393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=43661
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:51.169556 16581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:51.170823 16581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:51.183488 16591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:51.183706 16594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:51.183740 16592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:51.184859 16581 server_base.cc:1034] running on GCE node
I20260811 12:09:51.185408 16581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:51.186291 16581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:51.187501 16581 hybrid_clock.cc:648] HybridClock initialized: now 1786450191187456 us; error 31 us; skew 500 ppm
I20260811 12:09:51.187851 16581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:51.190954 16581 init.cc:377] Logged in from keytab as oryx/127.4.48.194@KRBTEST.COM (short username oryx)
I20260811 12:09:51.192462 16581 webserver.cc:460] Webserver started at http://127.4.48.194:43661/ using document root <none> and password file <none>
I20260811 12:09:51.193166 16581 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:51.193296 16581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:51.198606 16581 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:09:51.201146 16608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.202368 16581 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 12:09:51.202512 16581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "13fb3c5909d948cfb9c261038eed3f36"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "437444766b3d9c3b894eb24bff86e783"
server_key_iv: "87c8a318fc77928d543e6a94f7a06ce3"
server_key_version: "encryptionkey@0"
I20260811 12:09:51.203418 16581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:51.218039 16581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:51.221966 16581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:51.222214 16581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:51.223114 16581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:51.224447 16581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:51.224534 16581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.224637 16581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:51.224713 16581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.254587 16581 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:41393
I20260811 12:09:51.254652 16805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:41393 every 8 connection(s)
I20260811 12:09:51.255995 16581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:51.263723 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 16581
I20260811 12:09:51.264043 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 15405
I20260811 12:09:51.282485 16157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: NON_VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: true } } }
I20260811 12:09:51.289631 16157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.195@KRBTEST.COM
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:51.318583 16830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.283400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:49831 (local address 127.4.48.252:45923)
0811 12:09:51.297673 (+ 14273us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.297679 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.300716 (+ 3037us) server_negotiation.cc:407] Connection header received
0811 12:09:51.300858 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.300864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.300962 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.301078 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.302528 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.303719 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.305754 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.306417 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.306510 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.311057 (+ 4547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.311064 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.311070 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.311111 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.314702 (+ 3591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.315702 (+ 1000us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.315709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.315713 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.315861 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.316450 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.316456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.316460 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.316815 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.316934 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.317972 (+ 1038us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.318192 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14161,"thread_start_us":14287,"threads_started":1}
I20260811 12:09:51.318899 16823 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: attempting to promote NON_VOTER cfc348bd49744de08a3b76aecea07d98 to VOTER
I20260811 12:09:51.320190 16825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.283316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.4.48.194:49831)
0811 12:09:51.300597 (+ 17281us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.300615 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.300736 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.301309 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.301314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.301968 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.302339 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.302351 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.303909 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.303915 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.305571 (+ 1656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.305585 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.306907 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.306914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.307149 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.307841 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.307864 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.310795 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.314872 (+ 4077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.314882 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.314901 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.315528 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.316014 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.316020 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.316024 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.316283 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.317048 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.317057 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.317513 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.317870 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.318416 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17163,"thread_start_us":17045,"threads_started":1}
I20260811 12:09:51.319662 16823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:51.322480 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.265565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46510 (local address 127.4.48.195:44855)
0811 12:09:51.269047 (+ 3482us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.269057 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.269090 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:09:51.269165 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.269174 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.269439 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.269616 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.274589 (+ 4973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.276237 (+ 1648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.282810 (+ 6573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.283623 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.286889 (+ 3266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.310132 (+ 23243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.310148 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.310167 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.310219 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.314774 (+ 4555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.318416 (+ 3642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.318426 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.318429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.318592 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.320110 (+ 1518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.320116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.320119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.320611 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.320898 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.321760 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.322075 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3292,"thread_start_us":3102,"threads_started":1}
I20260811 12:09:51.322686 16806 heartbeater.cc:347] Connected to a master server at 127.4.48.252:45923
I20260811 12:09:51.323014 16806 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.323272 16246 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 LEARNER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:09:51.323944 16806 heartbeater.cc:511] Master 127.4.48.252:45923 requested a full tablet report, sending...
I20260811 12:09:51.324760 15627 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:51.324726 16157 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:09:51.325682 15901 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Refusing update from remote peer 6c85c904d2514636985e638d1f662f20: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 12:09:51.325709 16232 ts_manager.cc:194] Registered new tserver with Master: 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:51.327462 16823 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [LEADER]: Connected to new peer: Peer: permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 12:09:51.328902 16813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.258712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:45081 (local address 127.4.48.253:37519)
0811 12:09:51.278811 (+ 20099us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.278817 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.282724 (+ 3907us) server_negotiation.cc:407] Connection header received
0811 12:09:51.283962 (+ 1238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.283969 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.284059 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.284143 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.292536 (+ 8393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.293639 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.299976 (+ 6337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.300611 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.300701 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.316170 (+ 15469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.316177 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.316181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.316220 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.319606 (+ 3386us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.320431 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.320436 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.320439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.320581 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.326427 (+ 5846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.326433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.326436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.326785 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.326888 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.328346 (+ 1458us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.328573 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19945,"thread_start_us":19805,"threads_started":1}
I20260811 12:09:51.328820 16824 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfc348bd49744de08a3b76aecea07d98 (127.4.48.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.329855 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.259329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.4.48.194:45081)
0811 12:09:51.282625 (+ 23296us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.282643 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.283785 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.284737 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.284750 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.285734 (+ 984us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.286622 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.286647 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.293883 (+ 7236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.293889 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.296009 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.296026 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.301080 (+ 5054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.301088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.301379 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.304418 (+ 3039us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.304451 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.316001 (+ 11550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.319779 (+ 3778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.319784 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.319787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.320290 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.325506 (+ 5216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.325511 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.325515 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.325651 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.326952 (+ 1301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.326955 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.327066 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.329071 (+ 2005us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.329393 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23144,"mutex_wait_us":50,"spinlock_wait_cycles":17152,"thread_start_us":22850,"threads_started":1}
I20260811 12:09:51.329648 16246 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfc348bd49744de08a3b76aecea07d98 (127.4.48.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.330641 16809 heartbeater.cc:347] Connected to a master server at 127.4.48.253:37519
I20260811 12:09:51.330771 16809 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.330772 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.258692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:38407 (local address 127.4.48.254:33371)
0811 12:09:51.278372 (+ 19680us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.278380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.279370 (+ 990us) server_negotiation.cc:407] Connection header received
0811 12:09:51.296211 (+ 16841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.296217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.296311 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.296388 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.297535 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.298662 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.306719 (+ 8057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.307972 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.308074 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.319746 (+ 11672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.319752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.319757 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.319797 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.324723 (+ 4926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.325807 (+ 1084us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.325812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.325814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.325964 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.327254 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.327260 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.327263 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.327645 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.327798 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.330161 (+ 2363us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.330396 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19538,"thread_start_us":20136,"threads_started":1}
I20260811 12:09:51.331247 16809 heartbeater.cc:511] Master 127.4.48.253:37519 requested a full tablet report, sending...
I20260811 12:09:51.331693 16815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.259332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:38407)
0811 12:09:51.279202 (+ 19870us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.279259 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.281992 (+ 2733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.296589 (+ 14597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.296594 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.297068 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.297334 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.297345 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.298924 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.298930 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.302796 (+ 3866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.302813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.308437 (+ 5624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.308444 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.308636 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.316208 (+ 7572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.316228 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.319473 (+ 3245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.324930 (+ 5457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.324934 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.324938 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.325374 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.326098 (+ 724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.326102 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.326106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.326277 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.327879 (+ 1602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.327884 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.328046 (+ 162us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.330983 (+ 2937us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.331291 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19578,"thread_start_us":19320,"threads_started":1}
I20260811 12:09:51.332434 16808 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:51.332561 16808 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.332944 16808 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:51.333014 15886 ts_manager.cc:194] Registered new tserver with Master: 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:51.335186 15901 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, cfc348bd49744de08a3b76aecea07d98 (127.4.48.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.335738 14787 ts_manager.cc:194] Re-registered known tserver with Master: 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:51.336915 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:38407
I20260811 12:09:51.337431 16823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.337782 16823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:51.338094 16824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.338646 16157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.338938 16157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:51.339150 16824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:51.339948 16839 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.340376 16839 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:51.341233 16840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c85c904d2514636985e638d1f662f20" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:51.341614 16840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:51.419741 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 15405
I20260811 12:09:51.443791 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:44855
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:33371
--webserver_port=35685
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:09:51.714672 16558 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
W20260811 12:09:51.767578 16841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:51.767902 16841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:51.768007 16841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:51.775163 16841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:51.775256 16841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:51.775368 16841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:09:51.785022 16841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:44855
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=35685
--tserver_master_addrs=127.4.48.254:33371,127.4.48.253:37519,127.4.48.252:45923
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:51.786126 16841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:51.787287 16841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:51.799990 16850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:51.800114 16854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:51.800014 16851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:51.800829 16841 server_base.cc:1034] running on GCE node
I20260811 12:09:51.801366 16841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:51.802207 16841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:51.803412 16841 hybrid_clock.cc:648] HybridClock initialized: now 1786450191803373 us; error 29 us; skew 500 ppm
I20260811 12:09:51.803761 16841 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:51.806692 16841 init.cc:377] Logged in from keytab as oryx/127.4.48.195@KRBTEST.COM (short username oryx)
I20260811 12:09:51.808163 16841 webserver.cc:460] Webserver started at http://127.4.48.195:35685/ using document root <none> and password file <none>
I20260811 12:09:51.808800 16841 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:51.808934 16841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:51.814164 16841 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:09:51.816708 16866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.817950 16841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:09:51.818104 16841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "40e2907186cd4c979b7d12122180a8c7"
format_stamp: "Formatted at 2026-08-11 12:09:48 on dist-test-slave-nj21"
server_key: "7415f22c20265e44affb4dd3504bab86"
server_key_iv: "e027235c578a525894189be340596896"
server_key_version: "encryptionkey@0"
I20260811 12:09:51.819013 16841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:51.840461 16841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:51.844408 16841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:51.844641 16841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:51.845499 16841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:09:51.846829 16841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:51.846925 16841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.847036 16841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:51.847105 16841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:51.876885 16841 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44855
I20260811 12:09:51.876947 17062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44855 every 8 connection(s)
I20260811 12:09:51.878268 16841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:09:51.885156 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 16841
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.253@KRBTEST.COM
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
Aug 11 12:09:51 dist-test-slave-nj21 krb5kdc[14702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450191, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.252@KRBTEST.COM
I20260811 12:09:51.905738 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.881011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:37411 (local address 127.4.48.254:33371)
0811 12:09:51.881475 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.881481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.883113 (+ 1632us) server_negotiation.cc:407] Connection header received
0811 12:09:51.885085 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.885091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.885178 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.885271 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.888016 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889152 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892172 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.892811 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892914 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.898467 (+ 5553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.898473 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.898478 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.898517 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.902664 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903488 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.903494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.903497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.903656 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.904095 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.904099 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.904103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.904440 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.904562 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.905155 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.905381 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":221,"threads_started":1}
I20260811 12:09:51.906085 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.883229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:57515 (local address 127.4.48.252:45923)
0811 12:09:51.884007 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.884011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.884027 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:09:51.885099 (+ 1072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.885105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.885191 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.885299 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.888040 (+ 2741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889233 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892189 (+ 2956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.892876 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892979 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.898862 (+ 5883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.898869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.898875 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.898918 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.902249 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903217 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.903224 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.903228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.903388 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903901 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.903907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.903910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.904272 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.904414 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.905168 (+ 754us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.905408 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":579,"threads_started":1}
I20260811 12:09:51.906874 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.881593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:37411)
0811 12:09:51.882544 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.882605 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.884700 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.885651 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.885662 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.886674 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.887791 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.887822 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889373 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.889378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.891983 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.891998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.893374 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.893381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.893745 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.894562 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.894581 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.898102 (+ 3521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.902845 (+ 4743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.902849 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.902852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.903355 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.903819 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.903825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.903966 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.904645 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.904649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.904773 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.906004 (+ 1231us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.906253 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"mutex_wait_us":523,"spinlock_wait_cycles":76416,"thread_start_us":332,"threads_started":1}
I20260811 12:09:51.907470 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.881589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:37551 (local address 127.4.48.253:37519)
0811 12:09:51.882020 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.882026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.882888 (+ 862us) server_negotiation.cc:407] Connection header received
0811 12:09:51.885025 (+ 2137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.885031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.885119 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.885203 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.888006 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889152 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892171 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.892826 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.892932 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.898677 (+ 5745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.898683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.898688 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.898725 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.902007 (+ 3282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.902979 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.902985 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.902988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.903135 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903661 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.903666 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.903698 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.904066 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.904198 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.906900 (+ 2702us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.907117 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":214,"threads_started":1}
I20260811 12:09:51.907771 17080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.883089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.4.48.195:57515)
0811 12:09:51.883654 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.883689 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.884708 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.885656 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.885663 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.886682 (+ 1019us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.887843 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.887859 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889413 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.889419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.891965 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.891982 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.893374 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.893381 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.893758 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.894711 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.894746 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.898629 (+ 3883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.902418 (+ 3789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.902424 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.902428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.903069 (+ 641us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.903538 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.903547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.903771 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.904501 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.904506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.904708 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.906508 (+ 1800us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.906813 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":439,"spinlock_wait_cycles":33408,"thread_start_us":289,"threads_started":1}
I20260811 12:09:51.907871 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.881592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.4.48.195:37551)
0811 12:09:51.882544 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.882605 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.884700 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.885651 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.885662 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.886674 (+ 1012us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:51.887791 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.887828 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.889401 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.889406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.891983 (+ 2577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.891998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.893606 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.893612 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.893805 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.894541 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:51.894572 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:51.898460 (+ 3888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:51.902180 (+ 3720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.902189 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.902204 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.902812 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.903271 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:51.903280 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:51.903284 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:51.903525 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:51.904273 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:51.904281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:51.904707 (+ 426us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.905038 (+ 331us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.905509 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":420,"spinlock_wait_cycles":91776,"thread_start_us":474,"threads_started":1}
I20260811 12:09:51.908640 17064 heartbeater.cc:347] Connected to a master server at 127.4.48.254:33371
I20260811 12:09:51.908988 17064 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.909914 17064 heartbeater.cc:511] Master 127.4.48.254:33371 requested a full tablet report, sending...
I20260811 12:09:51.910624 17066 heartbeater.cc:347] Connected to a master server at 127.4.48.253:37519
I20260811 12:09:51.910761 17066 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.911175 17066 heartbeater.cc:511] Master 127.4.48.253:37519 requested a full tablet report, sending...
I20260811 12:09:51.911496 17063 heartbeater.cc:347] Connected to a master server at 127.4.48.252:45923
I20260811 12:09:51.911612 17063 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:51.912041 17063 heartbeater.cc:511] Master 127.4.48.252:45923 requested a full tablet report, sending...
I20260811 12:09:51.912815 14787 ts_manager.cc:194] Re-registered known tserver with Master: 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855)
I20260811 12:09:51.913954 14787 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:37411
I20260811 12:09:51.915161 15886 ts_manager.cc:194] Registered new tserver with Master: 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855)
I20260811 12:09:51.915345 16232 ts_manager.cc:194] Registered new tserver with Master: 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855)
I20260811 12:09:51.936431 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.911378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34824 (local address 127.4.48.253:37519)
0811 12:09:51.911534 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.911539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.920996 (+ 9457us) server_negotiation.cc:407] Connection header received
0811 12:09:51.921155 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.921160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.921232 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.921323 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.923257 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.924268 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.926049 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.926683 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.926779 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.930225 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.930229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.930233 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.930262 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.933015 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.934330 (+ 1315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.934336 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.934339 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.934485 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.934922 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.934928 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.934931 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.935280 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.935552 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.935879 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.936102 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 12:09:51.937868 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.920993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37376 (local address 127.4.48.252:45923)
0811 12:09:51.921149 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.921154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.923698 (+ 2544us) server_negotiation.cc:407] Connection header received
0811 12:09:51.923967 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.923973 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.924082 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.924187 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.926015 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.926988 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.928559 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.929124 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.929247 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.932236 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.932243 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.932247 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.932283 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.935193 (+ 2910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.935967 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.935972 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.935975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.936138 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.936513 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.936517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.936520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.936809 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.937048 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.937394 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.937610 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:51.937944 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.911098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.4.48.254:33371)
0811 12:09:51.916226 (+ 5128us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.916232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.920324 (+ 4092us) server_negotiation.cc:407] Connection header received
0811 12:09:51.920581 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.920587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.920685 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.920836 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:51.925550 (+ 4714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.926383 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.927940 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.928546 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.928672 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.932235 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:51.932240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:51.932245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:51.932275 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:51.934750 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.935493 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.935499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.935502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.935641 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:51.936110 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:51.936114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:51.936117 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:51.936461 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:51.936812 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.937431 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.937649 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5004}
I20260811 12:09:51.942159 14787 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60876:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:51.948835 14787 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:51.984292 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.965218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35003 (local address 127.0.0.1:60752)
0811 12:09:51.971881 (+ 6663us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.971904 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.972041 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.978754 (+ 6713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.978759 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.978798 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:51.979119 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.979128 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.980745 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.980750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.982172 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.982184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.983505 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.983511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.983700 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.983734 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.983827 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.983955 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":293,"threads_started":1}
I20260811 12:09:51.985266 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.965406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.4.48.193:35003)
0811 12:09:51.978240 (+ 12834us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.978249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.978281 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:09:51.978361 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.978367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.978573 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.978758 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:51.979266 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.980616 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.982322 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.983322 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.983443 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.983663 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.984778 (+ 1115us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.984955 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12680,"thread_start_us":12570,"threads_started":1}
I20260811 12:09:51.992344 16440 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (DEFAULT_TABLE table=test-table [id=9076e2642f9b4a1c994d91330991ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:51.993891 16440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca0a562a2f64984b374f17a7ba9e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:51.994113 17118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.965862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:44855 (local address 127.0.0.1:46514)
0811 12:09:51.977462 (+ 11600us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.977481 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.977601 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.985939 (+ 8338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.985944 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.985964 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:51.986352 (+ 388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.986364 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.988067 (+ 1703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.988073 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.990152 (+ 2079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.990167 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.992433 (+ 2266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.992457 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.992634 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.992667 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.992756 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.992887 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11468,"mutex_wait_us":21,"thread_start_us":22332,"threads_started":1}
I20260811 12:09:51.994160 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.966616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46514 (local address 127.4.48.195:44855)
0811 12:09:51.984220 (+ 17604us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.984227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.984261 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:09:51.984359 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.984366 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.984598 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.986520 (+ 1922us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:51.986760 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.987919 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.990689 (+ 2770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.992231 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.992352 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.992546 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.993648 (+ 1102us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.993820 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17436,"thread_start_us":17273,"threads_started":1}
I20260811 12:09:51.994536 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.966248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36722 (local address 127.4.48.194:41393)
0811 12:09:51.985135 (+ 18887us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:51.985142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:51.985168 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:09:51.985257 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:51.985264 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:51.985489 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:51.985677 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:51.987774 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.989138 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.991353 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.992658 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.992782 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.992966 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:51.993179 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:09:51.993343 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18731,"thread_start_us":18597,"threads_started":1}
I20260811 12:09:51.994551 17114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:51.966051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:41393 (local address 127.0.0.1:36722)
0811 12:09:51.984514 (+ 18463us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:51.984532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:51.984618 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:51.985801 (+ 1183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:51.985805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:51.985828 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:51.986010 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.986021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.989304 (+ 3283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.989310 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.991148 (+ 1838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:51.991163 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:51.992809 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:51.992815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:51.993002 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:51.993040 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:09:51.993122 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 12:09:51.993308 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18363,"mutex_wait_us":2,"thread_start_us":11631,"threads_started":1}
I20260811 12:09:51.998091 16694 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (DEFAULT_TABLE table=test-table [id=9076e2642f9b4a1c994d91330991ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:51.999434 16694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca0a562a2f64984b374f17a7ba9e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:52.001129 16945 tablet_service.cc:1467] Processing CreateTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (DEFAULT_TABLE table=test-table [id=9076e2642f9b4a1c994d91330991ca8a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:52.002257 16945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ca0a562a2f64984b374f17a7ba9e1e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:52.010223 17123 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Bootstrap starting.
I20260811 12:09:52.014151 17123 tablet_bootstrap.cc:654] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:52.014341 17124 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Bootstrap starting.
I20260811 12:09:52.015636 17123 log.cc:824] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:52.015775 17125 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Bootstrap starting.
I20260811 12:09:52.017673 17124 tablet_bootstrap.cc:654] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:52.018505 17123 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: No bootstrap required, opened a new log
I20260811 12:09:52.018741 17123 ts_tablet_manager.cc:1397] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 12:09:52.018878 17124 log.cc:824] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:52.019085 17125 tablet_bootstrap.cc:654] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:52.020242 17125 log.cc:824] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:52.021963 17124 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: No bootstrap required, opened a new log
I20260811 12:09:52.022233 17124 ts_tablet_manager.cc:1397] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260811 12:09:52.023223 17125 tablet_bootstrap.cc:492] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: No bootstrap required, opened a new log
I20260811 12:09:52.023480 17125 ts_tablet_manager.cc:1397] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:09:52.026844 17123 raft_consensus.cc:348] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.027173 17123 raft_consensus.cc:374] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:52.027277 17123 raft_consensus.cc:729] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9652be36c94c80b06d8c2d1d62f91e, State: Initialized, Role: FOLLOWER
I20260811 12:09:52.027853 17123 consensus_queue.cc:260] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.029497 17123 ts_tablet_manager.cc:1428] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 12:09:52.033339 17125 raft_consensus.cc:348] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.033763 17125 raft_consensus.cc:374] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:52.033871 17125 raft_consensus.cc:729] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e2907186cd4c979b7d12122180a8c7, State: Initialized, Role: FOLLOWER
I20260811 12:09:52.034427 17125 consensus_queue.cc:260] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.035483 17064 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
I20260811 12:09:52.036130 17125 ts_tablet_manager.cc:1428] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260811 12:09:52.036123 17124 raft_consensus.cc:348] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.037130 17124 raft_consensus.cc:374] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:52.037223 17124 raft_consensus.cc:729] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fb3c5909d948cfb9c261038eed3f36, State: Initialized, Role: FOLLOWER
I20260811 12:09:52.038244 17124 consensus_queue.cc:260] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.039330 16808 heartbeater.cc:503] Master 127.4.48.254:33371 was elected leader, sending a full tablet report...
I20260811 12:09:52.041296 17124 ts_tablet_manager.cc:1428] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Time spent starting tablet: real 0.019s user 0.010s sys 0.003s
W20260811 12:09:52.131131 17067 tablet.cc:2369] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:52.136337 17132 raft_consensus.cc:482] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:52.136636 17132 raft_consensus.cc:504] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.138209 17132 leader_election.cc:288] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855), 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:52.147046 17121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.138652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:59275 (local address 127.4.48.194:41393)
0811 12:09:52.138848 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.138854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.139717 (+ 863us) server_negotiation.cc:407] Connection header received
0811 12:09:52.139918 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.139924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.140001 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.140096 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:52.140771 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.141880 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.144843 (+ 2963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.146045 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.146178 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.146253 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.146663 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.146791 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:09:52.147046 17122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.138572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:45087 (local address 127.4.48.195:44855)
0811 12:09:52.138800 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.138806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.139717 (+ 911us) server_negotiation.cc:407] Connection header received
0811 12:09:52.139918 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.139924 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.140001 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.140093 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:52.140771 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.141870 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.144849 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.146024 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.146182 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.146253 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.146663 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.146792 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 12:09:52.147294 17138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.138685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:41393 (local address 127.4.48.193:59275)
0811 12:09:52.139584 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:52.139604 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:52.139787 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:52.140124 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:52.140129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:52.140165 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:52.140598 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:52.140611 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.142005 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.142011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:52.144690 (+ 2679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:52.144706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.146253 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.146259 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:52.146472 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.146525 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 12:09:52.146614 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:09:52.146779 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":175,"spinlock_wait_cycles":27008,"thread_start_us":511,"threads_started":1}
I20260811 12:09:52.147353 17137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.138660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:44855 (local address 127.4.48.193:45087)
0811 12:09:52.139593 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:52.139606 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:52.139787 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:52.140124 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:52.140129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:52.140162 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:52.140643 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:52.140657 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.142005 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.142011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:52.144690 (+ 2679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:52.144706 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.146289 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.146296 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:52.146488 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.146540 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:09:52.146614 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:09:52.146778 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":779,"mutex_wait_us":269,"spinlock_wait_cycles":7808,"thread_start_us":496,"threads_started":1}
I20260811 12:09:52.148280 16720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca0a562a2f64984b374f17a7ba9e1e7" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fb3c5909d948cfb9c261038eed3f36" is_pre_election: true
I20260811 12:09:52.148257 16985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca0a562a2f64984b374f17a7ba9e1e7" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e2907186cd4c979b7d12122180a8c7" is_pre_election: true
I20260811 12:09:52.148753 16985 raft_consensus.cc:2454] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 0.
I20260811 12:09:52.148754 16720 raft_consensus.cc:2454] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 0.
I20260811 12:09:52.149344 16351 leader_election.cc:302] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f9652be36c94c80b06d8c2d1d62f91e, 40e2907186cd4c979b7d12122180a8c7; no voters:
I20260811 12:09:52.149714 17132 raft_consensus.cc:2781] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:52.149838 17132 raft_consensus.cc:482] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:52.149935 17132 raft_consensus.cc:3037] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Advancing to term 1
W20260811 12:09:52.151083 16560 tablet.cc:2369] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:52.152484 17132 raft_consensus.cc:504] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.153090 17132 leader_election.cc:288] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 election: Requested vote from peers 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855), 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:52.153415 16985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca0a562a2f64984b374f17a7ba9e1e7" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e2907186cd4c979b7d12122180a8c7"
I20260811 12:09:52.153556 16720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ca0a562a2f64984b374f17a7ba9e1e7" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fb3c5909d948cfb9c261038eed3f36"
I20260811 12:09:52.153615 16985 raft_consensus.cc:3037] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:52.153754 16720 raft_consensus.cc:3037] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:52.156702 16985 raft_consensus.cc:2454] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 1.
I20260811 12:09:52.156803 16720 raft_consensus.cc:2454] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 1.
I20260811 12:09:52.157099 16351 leader_election.cc:302] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f9652be36c94c80b06d8c2d1d62f91e, 40e2907186cd4c979b7d12122180a8c7; no voters:
I20260811 12:09:52.157375 17132 raft_consensus.cc:2781] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:52.157769 17132 raft_consensus.cc:686] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 LEADER]: Becoming Leader. State: Replica: 0f9652be36c94c80b06d8c2d1d62f91e, State: Running, Role: LEADER
I20260811 12:09:52.158257 17132 consensus_queue.cc:237] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:52.163125 14785 catalog_manager.cc:5495] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193). New cstate: current_term: 1 leader_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } health_report { overall_health: UNKNOWN } } }
I20260811 12:09:52.209645 17115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.204490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60760 (local address 127.4.48.193:35003)
0811 12:09:52.204658 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.204662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.204789 (+ 127us) server_negotiation.cc:407] Connection header received
0811 12:09:52.204968 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.204972 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.205055 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.205163 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:52.205489 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.206181 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.207486 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.208024 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.208116 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.208541 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:52.209112 (+ 571us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:52.209216 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.209380 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.209456 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:09:52.217500 16985 raft_consensus.cc:1261] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Refusing update from remote peer 0f9652be36c94c80b06d8c2d1d62f91e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:52.217554 16720 raft_consensus.cc:1261] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Refusing update from remote peer 0f9652be36c94c80b06d8c2d1d62f91e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:52.218379 17139 consensus_queue.cc:1035] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:52.218760 17132 consensus_queue.cc:1035] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:52.232479 17145 mvcc.cc:205] Tried to move back new op lower bound from 7317299987313856512 to 7317299987083829248. Current Snapshot: MvccSnapshot[applied={T|T < 7317299987313856512}]
I20260811 12:09:52.250820 14785 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:52.251151 14785 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60876:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:52.267905 14785 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6c85c904d2514636985e638d1f662f20: Sending DeleteTablet for 3 replicas of tablet 9ca0a562a2f64984b374f17a7ba9e1e7
I20260811 12:09:52.269269 16945 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:52 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:46514
I20260811 12:09:52.269646 16694 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:52 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:36722
I20260811 12:09:52.269995 16942 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:52 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:46514
I20260811 12:09:52.270249 16692 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:52 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:36722
I20260811 12:09:52.270695 16440 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:52 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:60752
I20260811 12:09:52.278723 17162 tablet_replica.cc:330] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: stopping tablet replica
I20260811 12:09:52.279779 17163 tablet_replica.cc:330] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: stopping tablet replica
I20260811 12:09:52.280411 17162 raft_consensus.cc:2229] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:52.281268 17163 raft_consensus.cc:2229] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:52.281898 17162 raft_consensus.cc:2258] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:52.284330 17160 tablet_replica.cc:330] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: stopping tablet replica
I20260811 12:09:52.284714 17160 raft_consensus.cc:2229] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:52.285626 17160 raft_consensus.cc:2258] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:52.285655 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 14723
I20260811 12:09:52.286881 14843 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:52.287971 17160 ts_tablet_manager.cc:1905] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:52.290009 17163 raft_consensus.cc:2258] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:52.291203 17162 ts_tablet_manager.cc:1905] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:52.293618 17163 ts_tablet_manager.cc:1905] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:52.298540 17162 ts_tablet_manager.cc:1918] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:52.298707 17162 log.cc:1196] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/9ca0a562a2f64984b374f17a7ba9e1e7
I20260811 12:09:52.299312 17162 ts_tablet_manager.cc:1939] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 13fb3c5909d948cfb9c261038eed3f36: Deleting consensus metadata
I20260811 12:09:52.299714 17160 ts_tablet_manager.cc:1918] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:52.300007 17160 log.cc:1196] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/9ca0a562a2f64984b374f17a7ba9e1e7
I20260811 12:09:52.300669 17160 ts_tablet_manager.cc:1939] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 40e2907186cd4c979b7d12122180a8c7: Deleting consensus metadata
I20260811 12:09:52.307081 17163 ts_tablet_manager.cc:1918] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:52.307283 17163 log.cc:1196] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/9ca0a562a2f64984b374f17a7ba9e1e7
I20260811 12:09:52.307978 17163 ts_tablet_manager.cc:1939] T 9ca0a562a2f64984b374f17a7ba9e1e7 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting consensus metadata
I20260811 12:09:52.315412 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.273983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34840 (local address 127.4.48.253:37519)
0811 12:09:52.274150 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.274156 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.283653 (+ 9497us) server_negotiation.cc:407] Connection header received
0811 12:09:52.283882 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.283888 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.283970 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.284061 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:52.297857 (+ 13796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.299106 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.303552 (+ 4446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.304347 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.304458 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.308570 (+ 4112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:52.308577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:52.308582 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:52.308623 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:52.312403 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.313232 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.313238 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.313241 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.313419 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.313841 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.313847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.313850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.314226 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:52.314517 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.314822 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.315046 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:09:52.332116 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.284236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37382 (local address 127.4.48.252:45923)
0811 12:09:52.284405 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.284410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.315421 (+ 31011us) server_negotiation.cc:407] Connection header received
0811 12:09:52.315739 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.315746 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.315846 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.315923 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:52.317819 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.318939 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.321004 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.321678 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.321794 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.325352 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:52.325360 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:52.325366 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:52.325431 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:52.329082 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.329878 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.329884 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.329888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.330067 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.330456 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.330461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.330465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.330821 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:52.331183 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.331400 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.331613 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:09:52.425993 14765 catalog_manager.cc:4841] TS 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855): tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (table test-table [id=9076e2642f9b4a1c994d91330991ca8a]) successfully deleted
W20260811 12:09:52.426258 14765 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.48.195:44855, user_credentials={real_user=slave}} blocked reactor thread for 123923us
I20260811 12:09:52.426749 14764 catalog_manager.cc:4841] TS 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393): tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (table test-table [id=9076e2642f9b4a1c994d91330991ca8a]) successfully deleted
W20260811 12:09:52.426926 14764 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.4.48.194:41393, user_credentials={real_user=slave}} blocked reactor thread for 125684us
I20260811 12:09:52.427007 14765 catalog_manager.cc:4841] TS 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003): tablet 9ca0a562a2f64984b374f17a7ba9e1e7 (table test-table [id=9076e2642f9b4a1c994d91330991ca8a]) successfully deleted
W20260811 12:09:52.427345 14765 catalog_manager.cc:4808] TS 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855): delete failed for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 because the tablet was not found. No further retry: Not found: Tablet not found: 9ca0a562a2f64984b374f17a7ba9e1e7
W20260811 12:09:52.427354 14764 catalog_manager.cc:4808] TS 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393): delete failed for tablet 9ca0a562a2f64984b374f17a7ba9e1e7 because the tablet was not found. No further retry: Not found: Tablet not found: 9ca0a562a2f64984b374f17a7ba9e1e7
I20260811 12:09:52.427727 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 14723
I20260811 12:09:52.475080 17084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.458380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37396 (local address 127.4.48.252:45923)
0811 12:09:52.458582 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.458588 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.458605 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:09:52.458658 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.458663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.458728 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.458845 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:52.462573 (+ 3728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.463702 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.465203 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.465883 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.466004 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.468677 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:52.468683 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:52.468686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:52.468726 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:52.472048 (+ 3322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.472973 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.472978 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.472981 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.473140 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.473523 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.473528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.473531 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.473925 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:52.474167 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.474518 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.474738 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:09:52.475409 17073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:52.457130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34852 (local address 127.4.48.253:37519)
0811 12:09:52.457373 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:52.457379 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:52.458319 (+ 940us) server_negotiation.cc:407] Connection header received
0811 12:09:52.458609 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:52.458616 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:52.458703 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:52.458828 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:52.461258 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.462388 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.463631 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:52.464296 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:52.464432 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:52.468821 (+ 4389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:52.468827 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:52.468831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:52.468870 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:52.472189 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.472965 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.472971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.472974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.473147 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:52.473637 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:52.473643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:52.473646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:52.474010 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:52.474199 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:52.474775 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 12:09:52.474988 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:09:53.246485 17178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.243561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:51771)
0811 12:09:53.246190 (+ 2629us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.246311 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2533,"thread_start_us":2430,"threads_started":1}
W20260811 12:09:53.246989 16558 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:33371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.271113 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.270409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:51475)
0811 12:09:53.270811 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.270940 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":207,"threads_started":1}
W20260811 12:09:53.271479 17064 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:33371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.271508 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.270827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:47017)
0811 12:09:53.271230 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.271357 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":197,"threads_started":1}
W20260811 12:09:53.271890 16808 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:33371 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.766415 17182 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c85c904d2514636985e638d1f662f20)
I20260811 12:09:53.766777 17182 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:53.766875 17183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 6c85c904d2514636985e638d1f662f20)
I20260811 12:09:53.767098 17183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:53.768286 17182 leader_election.cc:288] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253:37519)
I20260811 12:09:53.774591 17183 leader_election.cc:288] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923)
I20260811 12:09:53.779803 17187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.768528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33118)
0811 12:09:53.779200 (+ 10672us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.779450 (+ 250us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10521,"thread_start_us":10362,"threads_started":1}
W20260811 12:09:53.780151 16204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.780682 17184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.768466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33122)
0811 12:09:53.780164 (+ 11698us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.780404 (+ 240us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11585,"mutex_wait_us":36,"thread_start_us":11397,"threads_started":1}
W20260811 12:09:53.782375 15855 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.789191 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.768466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:45923 (local address 127.0.0.1:37412)
0811 12:09:53.779509 (+ 11043us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.779530 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:53.779690 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:53.782320 (+ 2630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:53.782325 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:53.782354 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:53.782680 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:53.782694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.784539 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.784544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:53.786672 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:53.786687 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.788431 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.788438 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:53.788602 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:53.788641 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 12:09:53.788713 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:09:53.788843 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10904,"thread_start_us":10744,"threads_started":1}
I20260811 12:09:53.790279 17190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.769135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37412 (local address 127.4.48.252:45923)
0811 12:09:53.781349 (+ 12214us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:53.781356 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:53.781377 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:09:53.781449 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:53.781455 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:53.781567 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:53.782847 (+ 1280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:53.783082 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.784394 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.786904 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.788256 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.789536 (+ 1280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:53.789670 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:53.789725 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 12:09:53.789855 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":12070,"thread_start_us":11948,"threads_started":1}
I20260811 12:09:53.790750 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.768920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:37519 (local address 127.0.0.1:34860)
0811 12:09:53.782031 (+ 13111us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.782049 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:53.782152 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:53.783220 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:53.783228 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:53.783252 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:53.783598 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:53.783612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.785250 (+ 1638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.785255 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:53.787485 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:53.787500 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.789375 (+ 1875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.789381 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:53.790015 (+ 634us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:53.790073 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:09:53.790187 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 12:09:53.790334 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13003,"mutex_wait_us":410,"thread_start_us":12877,"threads_started":1}
I20260811 12:09:53.790819 17191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.780173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34860 (local address 127.4.48.253:37519)
0811 12:09:53.782798 (+ 2625us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:53.782803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:53.782818 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:09:53.782866 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:53.782870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:53.782928 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:53.783015 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:53.783830 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.785085 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.787646 (+ 2561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:53.789042 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:53.789152 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:53.789264 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:53.790359 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0811 12:09:53.790489 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2491,"thread_start_us":1947,"threads_started":1}
I20260811 12:09:53.791030 16246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54629b4f2e5849189c73a42cf79c1a02" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc348bd49744de08a3b76aecea07d98" is_pre_election: true
I20260811 12:09:53.791433 16246 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54629b4f2e5849189c73a42cf79c1a02 in term 1.
I20260811 12:09:53.791502 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.791157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33126)
0811 12:09:53.791279 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.791372 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 12:09:53.791786 16204 leader_election.cc:334] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.791811 15901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc348bd49744de08a3b76aecea07d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54629b4f2e5849189c73a42cf79c1a02" is_pre_election: true
I20260811 12:09:53.792147 15901 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cfc348bd49744de08a3b76aecea07d98 in term 1.
I20260811 12:09:53.792243 15857 leader_election.cc:302] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54629b4f2e5849189c73a42cf79c1a02, cfc348bd49744de08a3b76aecea07d98; no voters:
I20260811 12:09:53.792678 17183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:09:53.792723 16205 leader_election.cc:302] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 54629b4f2e5849189c73a42cf79c1a02, cfc348bd49744de08a3b76aecea07d98; no voters: 6c85c904d2514636985e638d1f662f20
I20260811 12:09:53.792810 17183 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c85c904d2514636985e638d1f662f20)
I20260811 12:09:53.792841 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.792355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33138)
0811 12:09:53.792599 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.792696 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 12:09:53.792923 17183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:09:53.793308 17182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 12:09:53.793435 17182 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 6c85c904d2514636985e638d1f662f20)
I20260811 12:09:53.793529 17182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 1 FOLLOWER]: Advancing to term 2
W20260811 12:09:53.793045 15855 leader_election.cc:334] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.795483 17183 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:53.795886 17182 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:53.796204 17183 leader_election.cc:288] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 election: Requested vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923)
I20260811 12:09:53.796515 17182 leader_election.cc:288] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 election: Requested vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), 54629b4f2e5849189c73a42cf79c1a02 (127.4.48.253:37519)
I20260811 12:09:53.796702 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.796335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33150)
0811 12:09:53.796459 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.796557 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 12:09:53.797355 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.796990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33158)
0811 12:09:53.797114 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.797212 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 12:09:53.797501 15901 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cfc348bd49744de08a3b76aecea07d98" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "54629b4f2e5849189c73a42cf79c1a02"
I20260811 12:09:53.797896 15901 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate cfc348bd49744de08a3b76aecea07d98 in current term 2: Already voted for candidate 54629b4f2e5849189c73a42cf79c1a02 in this term.
I20260811 12:09:53.797942 16246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54629b4f2e5849189c73a42cf79c1a02" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc348bd49744de08a3b76aecea07d98"
I20260811 12:09:53.798213 16246 raft_consensus.cc:2379] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 54629b4f2e5849189c73a42cf79c1a02 in current term 2: Already voted for candidate cfc348bd49744de08a3b76aecea07d98 in this term.
I20260811 12:09:53.798640 17189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.798242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33162)
0811 12:09:53.798433 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.798511 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
W20260811 12:09:53.798902 16204 leader_election.cc:334] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.799083 17185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:53.798634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:33168)
0811 12:09:53.798858 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:53.798943 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 12:09:53.799111 16205 leader_election.cc:302] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cfc348bd49744de08a3b76aecea07d98; no voters: 54629b4f2e5849189c73a42cf79c1a02, 6c85c904d2514636985e638d1f662f20
W20260811 12:09:53.799292 15855 leader_election.cc:334] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:53.799391 17182 raft_consensus.cc:2726] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:09:53.799415 15855 leader_election.cc:302] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 54629b4f2e5849189c73a42cf79c1a02; no voters: 6c85c904d2514636985e638d1f662f20, cfc348bd49744de08a3b76aecea07d98
I20260811 12:09:53.799707 17183 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 12:09:54.248225 17201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:54.247567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:40087)
0811 12:09:54.247984 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:54.248063 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":235,"threads_started":1}
I20260811 12:09:54.272930 17202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:54.272112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:40601)
0811 12:09:54.272585 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:54.272704 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":243,"threads_started":1}
I20260811 12:09:54.273195 17204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:54.272427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:47425)
0811 12:09:54.272901 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:54.272996 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":266,"threads_started":1}
I20260811 12:09:55.249709 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.249018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:45671)
0811 12:09:55.249433 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.249529 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":194,"threads_started":1}
I20260811 12:09:55.274461 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.273759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:42737)
0811 12:09:55.274223 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.274303 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":267,"threads_started":1}
I20260811 12:09:55.274662 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.273954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:44297)
0811 12:09:55.274411 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.274491 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":246,"threads_started":1}
I20260811 12:09:55.434736 17208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:55.434944 17208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:55.435523 17208 leader_election.cc:288] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923)
I20260811 12:09:55.436076 16246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54629b4f2e5849189c73a42cf79c1a02" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc348bd49744de08a3b76aecea07d98" is_pre_election: true
I20260811 12:09:55.436331 16246 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54629b4f2e5849189c73a42cf79c1a02 in term 2.
I20260811 12:09:55.436652 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.435745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60088)
0811 12:09:55.436347 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.436439 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":364,"threads_started":1}
I20260811 12:09:55.436764 15857 leader_election.cc:302] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54629b4f2e5849189c73a42cf79c1a02, cfc348bd49744de08a3b76aecea07d98; no voters:
I20260811 12:09:55.437031 17208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 12:09:55.437188 17208 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:55.437296 17208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:09:55.438124 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.437750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60096)
0811 12:09:55.437894 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.437969 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:09:55.438342 15855 leader_election.cc:334] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:55.439855 17208 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:55.440375 17208 leader_election.cc:288] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 election: Requested vote from peers 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371), cfc348bd49744de08a3b76aecea07d98 (127.4.48.252:45923)
I20260811 12:09:55.440801 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.440488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60112)
0811 12:09:55.440595 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.440660 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260811 12:09:55.440826 16246 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "54629b4f2e5849189c73a42cf79c1a02" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "cfc348bd49744de08a3b76aecea07d98"
I20260811 12:09:55.441033 16246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:09:55.442960 16246 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54629b4f2e5849189c73a42cf79c1a02 in term 3.
I20260811 12:09:55.443341 15857 leader_election.cc:302] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54629b4f2e5849189c73a42cf79c1a02, cfc348bd49744de08a3b76aecea07d98; no voters:
I20260811 12:09:55.443603 17208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:09:55.444018 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.443609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60122)
0811 12:09:55.443767 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.443848 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:09:55.444016 17208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [term 3 LEADER]: Becoming Leader. State: Replica: 54629b4f2e5849189c73a42cf79c1a02, State: Running, Role: LEADER
W20260811 12:09:55.444247 15855 leader_election.cc:334] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
I20260811 12:09:55.444453 17208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } }
I20260811 12:09:55.446208 17213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54629b4f2e5849189c73a42cf79c1a02. Latest consensus state: current_term: 3 leader_uuid: "54629b4f2e5849189c73a42cf79c1a02" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:55.446507 17213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:55.447095 17215 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:55.451118 17215 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:55.451956 17215 catalog_manager.cc:1259] Loaded cluster ID: f90e8db32f08402080f23baa79fe4c52
I20260811 12:09:55.452035 17215 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:55.453141 17215 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:55.453990 17215 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Loaded TSK: 0
I20260811 12:09:55.454850 17215 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:09:55.501538 15883 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:09:55.508419 15883 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:09:55.512384 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.511999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60134)
0811 12:09:55.512136 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.512220 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:09:55.512579 16246 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [term 3 FOLLOWER]: Refusing update from remote peer 54629b4f2e5849189c73a42cf79c1a02: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 12:09:55.513201 17213 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 12:09:55.513341 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.512967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60146)
0811 12:09:55.513119 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.513200 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 12:09:55.513564 15855 consensus_peers.cc:487] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 -> Peer 6c85c904d2514636985e638d1f662f20 (127.4.48.254:33371): Couldn't send request to peer 6c85c904d2514636985e638d1f662f20. Status: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 12:09:55.515532 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54629b4f2e5849189c73a42cf79c1a02. Latest consensus state: current_term: 3 leader_uuid: "54629b4f2e5849189c73a42cf79c1a02" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:55.515826 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:55.517218 17213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "54629b4f2e5849189c73a42cf79c1a02" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:55.517426 17213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:55.518267 17218 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "54629b4f2e5849189c73a42cf79c1a02" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "6c85c904d2514636985e638d1f662f20" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 33371 } } peers { permanent_uuid: "54629b4f2e5849189c73a42cf79c1a02" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 37519 } attrs { promote: false } } peers { permanent_uuid: "cfc348bd49744de08a3b76aecea07d98" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 45923 } attrs { promote: false } } }
I20260811 12:09:55.518555 17218 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfc348bd49744de08a3b76aecea07d98 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:09:55.533387 17209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.525460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:44855 (local address 127.0.0.1:40026)
0811 12:09:55.525613 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.525632 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:55.525759 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:55.527120 (+ 1361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:55.527124 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:55.527154 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:55.527491 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.527505 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.529077 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.529083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.531296 (+ 2213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.531312 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.532875 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.532879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.533004 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.533030 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:09:55.533083 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 12:09:55.533168 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:09:55.534191 17236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.525849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35003 (local address 127.0.0.1:44666)
0811 12:09:55.526424 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.526440 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:55.526544 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:55.528826 (+ 2282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:55.528831 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:55.528857 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:55.529198 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.529208 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.530201 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.530206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.531817 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.531833 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.533475 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.533482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.533644 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.533675 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:09:55.533761 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:09:55.533880 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":319,"threads_started":1}
I20260811 12:09:55.534412 17231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.525546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.4.48.195:44855)
0811 12:09:55.526747 (+ 1201us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:55.526753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:55.526774 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:09:55.526860 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:55.526866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:55.526953 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:55.527642 (+ 689us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:55.527911 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.528875 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.531480 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.532676 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.533839 (+ 1163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.533943 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:55.533996 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 12:09:55.534121 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":864,"threads_started":1}
I20260811 12:09:55.534857 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.526245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.4.48.193:35003)
0811 12:09:55.528493 (+ 2248us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:55.528499 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:55.528520 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:09:55.528596 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:55.528602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:55.528677 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:55.528785 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:55.529366 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.530062 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.531981 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.533267 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.533396 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.533503 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:55.534401 (+ 898us) server_negotiation.cc:299] Negotiation successful
0811 12:09:55.534552 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2114,"thread_start_us":2129,"threads_started":1}
I20260811 12:09:55.535215 16945 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (DEFAULT_TABLE table=test-table [id=02ead65cd3104f0c930aae4155e1a8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:55.535524 16440 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (DEFAULT_TABLE table=test-table [id=02ead65cd3104f0c930aae4155e1a8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:55.535748 16945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:55.536017 16440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:55.541541 17243 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Bootstrap starting.
I20260811 12:09:55.543033 17243 tablet_bootstrap.cc:654] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:55.545661 17241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.526732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36878 (local address 127.4.48.194:41393)
0811 12:09:55.538620 (+ 11888us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:55.538626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:55.538646 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:09:55.538716 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:55.538721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:55.538794 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:55.538899 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:09:55.539600 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.540725 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.543101 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.544291 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.544444 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.544552 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:55.545141 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 12:09:55.545298 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11753,"thread_start_us":11644,"threads_started":1}
I20260811 12:09:55.546379 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.526657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:41393 (local address 127.0.0.1:36878)
0811 12:09:55.527292 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.527313 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:55.527431 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:55.538929 (+ 11498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:55.538934 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:55.538960 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:09:55.539275 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.539288 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.540865 (+ 1577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.540871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.542911 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:55.542928 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.544725 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.544734 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:55.544945 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.544981 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:09:55.545884 (+ 903us) client_negotiation.cc:240] Negotiation successful
0811 12:09:55.546036 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":341,"threads_started":1}
I20260811 12:09:55.547281 16694 tablet_service.cc:1467] Processing CreateTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (DEFAULT_TABLE table=test-table [id=02ead65cd3104f0c930aae4155e1a8f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:09:55.547801 16694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:55.550194 17244 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Bootstrap starting.
I20260811 12:09:55.551316 17243 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: No bootstrap required, opened a new log
I20260811 12:09:55.551466 17243 ts_tablet_manager.cc:1397] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.004s
I20260811 12:09:55.552145 17243 raft_consensus.cc:348] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.552336 17243 raft_consensus.cc:374] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:55.552445 17243 raft_consensus.cc:729] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f9652be36c94c80b06d8c2d1d62f91e, State: Initialized, Role: FOLLOWER
I20260811 12:09:55.552675 17243 consensus_queue.cc:260] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.553520 17243 ts_tablet_manager.cc:1428] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:09:55.556908 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.556515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:41605)
0811 12:09:55.556725 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.556795 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 12:09:55.558053 17244 tablet_bootstrap.cc:654] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:55.560055 17244 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: No bootstrap required, opened a new log
I20260811 12:09:55.560160 17248 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Bootstrap starting.
I20260811 12:09:55.560271 17244 ts_tablet_manager.cc:1397] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.001s
I20260811 12:09:55.561079 17244 raft_consensus.cc:348] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.561374 17244 raft_consensus.cc:374] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:55.561509 17244 raft_consensus.cc:729] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40e2907186cd4c979b7d12122180a8c7, State: Initialized, Role: FOLLOWER
I20260811 12:09:55.561923 17248 tablet_bootstrap.cc:654] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:55.561765 17244 consensus_queue.cc:260] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.562585 17244 ts_tablet_manager.cc:1428] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:09:55.564028 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.563160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:47225)
0811 12:09:55.563799 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.563888 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496}
I20260811 12:09:55.567337 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.566883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:46141)
0811 12:09:55.567108 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.567190 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 12:09:55.567842 17066 heartbeater.cc:503] Master 127.4.48.253:37519 was elected leader, sending a full tablet report...
I20260811 12:09:55.568955 17248 tablet_bootstrap.cc:492] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: No bootstrap required, opened a new log
I20260811 12:09:55.569092 17248 ts_tablet_manager.cc:1397] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260811 12:09:55.569705 17248 raft_consensus.cc:348] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.569887 17248 raft_consensus.cc:374] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:55.569983 17248 raft_consensus.cc:729] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13fb3c5909d948cfb9c261038eed3f36, State: Initialized, Role: FOLLOWER
I20260811 12:09:55.570220 17248 consensus_queue.cc:260] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.571031 17248 ts_tablet_manager.cc:1428] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 12:09:55.571755 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.571271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:57017)
0811 12:09:55.571488 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.571575 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 12:09:55.572796 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.572416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:44063)
0811 12:09:55.572563 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.572645 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:09:55.572860 16809 heartbeater.cc:503] Master 127.4.48.253:37519 was elected leader, sending a full tablet report...
W20260811 12:09:55.762202 16810 tablet.cc:2369] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:09:55.886503 17246 raft_consensus.cc:482] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:09:55.886746 17246 raft_consensus.cc:504] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.887512 17246 leader_election.cc:288] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855), 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:55.888034 16985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3e5a2b5f9a47b0bd959ba7c61c0975" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e2907186cd4c979b7d12122180a8c7" is_pre_election: true
I20260811 12:09:55.888118 16720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3e5a2b5f9a47b0bd959ba7c61c0975" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fb3c5909d948cfb9c261038eed3f36" is_pre_election: true
I20260811 12:09:55.888257 16985 raft_consensus.cc:2454] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 0.
I20260811 12:09:55.888381 16720 raft_consensus.cc:2454] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 0.
I20260811 12:09:55.888752 16351 leader_election.cc:302] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f9652be36c94c80b06d8c2d1d62f91e, 40e2907186cd4c979b7d12122180a8c7; no voters:
I20260811 12:09:55.889055 17246 raft_consensus.cc:2781] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:09:55.889176 17246 raft_consensus.cc:482] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:09:55.889268 17246 raft_consensus.cc:3037] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:55.891296 17246 raft_consensus.cc:504] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.892004 17246 leader_election.cc:288] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 election: Requested vote from peers 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855), 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393)
I20260811 12:09:55.892279 16985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3e5a2b5f9a47b0bd959ba7c61c0975" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "40e2907186cd4c979b7d12122180a8c7"
I20260811 12:09:55.892341 16720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b3e5a2b5f9a47b0bd959ba7c61c0975" candidate_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13fb3c5909d948cfb9c261038eed3f36"
I20260811 12:09:55.892457 16985 raft_consensus.cc:3037] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:55.892565 16720 raft_consensus.cc:3037] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:55.894977 16985 raft_consensus.cc:2454] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 1.
I20260811 12:09:55.895027 16720 raft_consensus.cc:2454] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f9652be36c94c80b06d8c2d1d62f91e in term 1.
I20260811 12:09:55.895349 16351 leader_election.cc:302] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f9652be36c94c80b06d8c2d1d62f91e, 40e2907186cd4c979b7d12122180a8c7; no voters:
I20260811 12:09:55.895644 17246 raft_consensus.cc:2781] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:55.896075 17246 raft_consensus.cc:686] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 LEADER]: Becoming Leader. State: Replica: 0f9652be36c94c80b06d8c2d1d62f91e, State: Running, Role: LEADER
I20260811 12:09:55.896358 16559 heartbeater.cc:503] Master 127.4.48.253:37519 was elected leader, sending a full tablet report...
I20260811 12:09:55.896369 17246 consensus_queue.cc:237] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } }
I20260811 12:09:55.897382 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.896882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:52987)
0811 12:09:55.897091 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.897187 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 12:09:55.899278 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.898893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:49855)
0811 12:09:55.899050 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:55.899134 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:09:55.899536 15883 catalog_manager.cc:5495] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193). New cstate: current_term: 1 leader_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f9652be36c94c80b06d8c2d1d62f91e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35003 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 } health_report { overall_health: UNKNOWN } } }
I20260811 12:09:55.993569 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.988904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44670 (local address 127.4.48.193:35003)
0811 12:09:55.989108 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:55.989113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:55.989133 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:09:55.989286 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:55.989290 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:55.989356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:55.989453 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:09:55.989836 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.990544 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.991872 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:55.992343 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:55.992530 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:55.992895 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:09:55.992999 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:09:55.993072 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:55.993238 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 12:09:55.993315 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:55.996558 16720 raft_consensus.cc:1261] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Refusing update from remote peer 0f9652be36c94c80b06d8c2d1d62f91e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:55.996608 16985 raft_consensus.cc:1261] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Refusing update from remote peer 0f9652be36c94c80b06d8c2d1d62f91e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:09:55.997213 17254 consensus_queue.cc:1035] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Connected to new peer: Peer: permanent_uuid: "13fb3c5909d948cfb9c261038eed3f36" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 41393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:55.997617 17246 consensus_queue.cc:1035] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [LEADER]: Connected to new peer: Peer: permanent_uuid: "40e2907186cd4c979b7d12122180a8c7" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:09:56.000641 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.000186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:51405)
0811 12:09:56.000359 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.000470 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:09:56.000741 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:55.999787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:54527)
0811 12:09:56.000456 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.000558 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531}
I20260811 12:09:56.017354 17205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.016800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.193:44947)
0811 12:09:56.017046 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.017143 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123}
I20260811 12:09:56.018682 17206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.016365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.195:53877)
0811 12:09:56.018421 (+ 2056us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.018513 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1936}
I20260811 12:09:56.020569 17207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.019966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.4.48.194:34963)
0811 12:09:56.020248 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.020355 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 12:09:56.024940 15883 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:56.025203 15883 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:09:56.028309 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.027891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60152)
0811 12:09:56.028054 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.028165 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 12:09:56.031394 15883 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54629b4f2e5849189c73a42cf79c1a02: Sending DeleteTablet for 3 replicas of tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975
I20260811 12:09:56.032581 16440 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:56 UTC) from {username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:44666
I20260811 12:09:56.032836 16945 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:56 UTC) from {username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:40026
I20260811 12:09:56.032830 16694 tablet_service.cc:1514] Processing DeleteTablet for tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:09:56 UTC) from {username='oryx', principal='oryx/127.4.48.253@KRBTEST.COM'} at 127.0.0.1:36878
I20260811 12:09:56.033627 17239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.033013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:33371 (local address 127.0.0.1:60154)
0811 12:09:56.033383 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:56.033468 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:33371: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265}
I20260811 12:09:56.034897 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 16193
I20260811 12:09:56.037377 17280 tablet_replica.cc:330] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: stopping tablet replica
I20260811 12:09:56.037523 17281 tablet_replica.cc:330] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: stopping tablet replica
I20260811 12:09:56.037586 17280 raft_consensus.cc:2229] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:09:56.037699 17281 raft_consensus.cc:2229] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:56.037937 17281 raft_consensus.cc:2258] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:56.038044 17280 raft_consensus.cc:2258] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:56.038888 17281 ts_tablet_manager.cc:1905] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:56.039232 17280 ts_tablet_manager.cc:1905] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:56.043394 17281 ts_tablet_manager.cc:1918] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:56.043514 17281 log.cc:1196] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/3b3e5a2b5f9a47b0bd959ba7c61c0975
I20260811 12:09:56.044046 17281 ts_tablet_manager.cc:1939] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 40e2907186cd4c979b7d12122180a8c7: Deleting consensus metadata
I20260811 12:09:56.045169 15861 catalog_manager.cc:4841] TS 40e2907186cd4c979b7d12122180a8c7 (127.4.48.195:44855): tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (table test-table [id=02ead65cd3104f0c930aae4155e1a8f6]) successfully deleted
I20260811 12:09:56.046000 17280 ts_tablet_manager.cc:1918] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:56.046169 17280 log.cc:1196] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/3b3e5a2b5f9a47b0bd959ba7c61c0975
I20260811 12:09:56.046662 17280 ts_tablet_manager.cc:1939] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 0f9652be36c94c80b06d8c2d1d62f91e: Deleting consensus metadata
I20260811 12:09:56.046705 17282 tablet_replica.cc:330] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: stopping tablet replica
I20260811 12:09:56.046941 17282 raft_consensus.cc:2229] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:09:56.047235 17282 raft_consensus.cc:2258] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:09:56.047704 15861 catalog_manager.cc:4841] TS 0f9652be36c94c80b06d8c2d1d62f91e (127.4.48.193:35003): tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (table test-table [id=02ead65cd3104f0c930aae4155e1a8f6]) successfully deleted
I20260811 12:09:56.049432 17282 ts_tablet_manager.cc:1905] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:09:56.054723 17282 ts_tablet_manager.cc:1918] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:09:56.054852 17282 log.cc:1196] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/3b3e5a2b5f9a47b0bd959ba7c61c0975
I20260811 12:09:56.055306 17282 ts_tablet_manager.cc:1939] T 3b3e5a2b5f9a47b0bd959ba7c61c0975 P 13fb3c5909d948cfb9c261038eed3f36: Deleting consensus metadata
I20260811 12:09:56.056344 15860 catalog_manager.cc:4841] TS 13fb3c5909d948cfb9c261038eed3f36 (127.4.48.194:41393): tablet 3b3e5a2b5f9a47b0bd959ba7c61c0975 (table test-table [id=02ead65cd3104f0c930aae4155e1a8f6]) successfully deleted
I20260811 12:09:56.058085 17238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.036877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44674 (local address 127.4.48.193:35003)
0811 12:09:56.037045 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:56.037051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:56.044075 (+ 7024us) server_negotiation.cc:407] Connection header received
0811 12:09:56.044409 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:56.044414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:56.044499 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:56.044571 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:56.046370 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.047453 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.048730 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.049286 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.049420 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:56.051840 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:56.051860 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:56.051883 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:56.051928 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:56.055584 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.056310 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.056320 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.056324 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.056496 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.056881 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.056884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.056886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.057239 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:56.057450 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:56.057684 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 12:09:56.057850 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:09:56.058672 16543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:56.174749 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 16193
I20260811 12:09:56.205960 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 16581
I20260811 12:09:56.218905 17286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.206774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36884 (local address 127.4.48.194:41393)
0811 12:09:56.207205 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:56.207210 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:56.207225 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:09:56.207276 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:56.207280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:56.207353 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:56.207447 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:56.208827 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.209565 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.210778 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.211178 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.211289 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:56.213684 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:56.213698 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:56.213717 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:56.213754 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:56.216307 (+ 2553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.216945 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.216953 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.216955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.217092 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.217441 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.217444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.217446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.217777 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:56.218060 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:56.218356 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 12:09:56.218566 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":211,"threads_started":1}
I20260811 12:09:56.219522 16796 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:56.309521 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 16581
I20260811 12:09:56.332768 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 16841
I20260811 12:09:56.346010 17292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:56.333505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40028 (local address 127.4.48.195:44855)
0811 12:09:56.333983 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:56.333988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:56.334005 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:09:56.334059 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:56.334063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:56.334138 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:56.334237 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:56.335511 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.336254 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.337437 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:56.337860 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:56.337949 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:56.340487 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:56.340502 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:56.340521 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:56.340551 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:56.343235 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.344060 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.344068 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.344071 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.344249 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:56.344644 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:56.344648 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:56.344651 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:56.345029 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:56.345250 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:56.345475 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 12:09:56.345662 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260811 12:09:56.346663 17056 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:56.436753 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 16841
I20260811 12:09:56.458735 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 15637
I20260811 12:09:56.459846 15943 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:56.578188 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 15637
I20260811 12:09:56.603780 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 15810
I20260811 12:09:56.604831 16284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:09:56.717907 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 15810
2026-08-11T12:09:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (11520 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:09:56 dist-test-slave-nj21 krb5kdc[17304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:09:56 dist-test-slave-nj21 krb5kdc[17304](info): set up 2 sockets
Aug 11 12:09:56 dist-test-slave-nj21 krb5kdc[17304](info): commencing operation
krb5kdc: starting...
W20260811 12:09:57.944212 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.175s user 0.007s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:09:58 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:09:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:09:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab.
Entry for principal oryx/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab.
I20260811 12:09:58.168463 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36375
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab
--principal=oryx/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:36375
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:58.485889 17322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:58.486181 17322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:58.486315 17322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:58.493265 17322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:09:58.493371 17322 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 12:09:58.493434 17322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:58.493517 17322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:58.493585 17322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:09:58.493669 17322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:09:58.503247 17322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:36375
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.254.keytab
--principal=oryx/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:58.504479 17322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:58.505748 17322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:58.517033 17334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:58.517071 17337 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:58.517096 17335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:58.517879 17322 server_base.cc:1034] running on GCE node
I20260811 12:09:58.518540 17322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:58.519819 17322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:58.521037 17322 hybrid_clock.cc:648] HybridClock initialized: now 1786450198520976 us; error 40 us; skew 500 ppm
I20260811 12:09:58.521344 17322 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:58 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:58.524327 17322 init.cc:377] Logged in from keytab as oryx/127.4.48.254@KRBTEST.COM (short username oryx)
I20260811 12:09:58.525688 17322 webserver.cc:460] Webserver started at http://127.4.48.254:43225/ using document root <none> and password file <none>
I20260811 12:09:58.526333 17322 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:58.526417 17322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:58.526690 17322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:58.529575 17322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "887d1b9d93d242f982cb0516898e5849"
format_stamp: "Formatted at 2026-08-11 12:09:58 on dist-test-slave-nj21"
server_key: "ceaf2793870275818fe960106399d0ca"
server_key_iv: "ea1b0dfc6cbfba7f2ecea9c7cc33a8e4"
server_key_version: "encryptionkey@0"
I20260811 12:09:58.530315 17322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "887d1b9d93d242f982cb0516898e5849"
format_stamp: "Formatted at 2026-08-11 12:09:58 on dist-test-slave-nj21"
server_key: "ceaf2793870275818fe960106399d0ca"
server_key_iv: "ea1b0dfc6cbfba7f2ecea9c7cc33a8e4"
server_key_version: "encryptionkey@0"
I20260811 12:09:58.535079 17322 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:09:58.538435 17346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:58.539638 17322 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:09:58.539819 17322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "887d1b9d93d242f982cb0516898e5849"
format_stamp: "Formatted at 2026-08-11 12:09:58 on dist-test-slave-nj21"
server_key: "ceaf2793870275818fe960106399d0ca"
server_key_iv: "ea1b0dfc6cbfba7f2ecea9c7cc33a8e4"
server_key_version: "encryptionkey@0"
I20260811 12:09:58.540006 17322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:58.554533 17322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:58.558173 17322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:58.558375 17322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:58.578171 17322 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:36375
I20260811 12:09:58.578178 17464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:36375 every 8 connection(s)
I20260811 12:09:58.579439 17322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:09:58.583011 17465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:09:58.587319 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 17322
I20260811 12:09:58.587508 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:09:58.588192 4291 external_mini_cluster.cc:1428] Setting key e4850db9ad285faba5c34a3a49b3fae0
I20260811 12:09:58.594739 17465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Bootstrap starting.
I20260811 12:09:58.598307 17465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Neither blocks nor log segments found. Creating new log.
I20260811 12:09:58.599443 17465 log.cc:824] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Log is configured to *not* fsync() on all Append() calls
I20260811 12:09:58.602227 17465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: No bootstrap required, opened a new log
Aug 11 12:09:58 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450198, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:58.612202 17470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:58.590012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.4.48.254:36375)
0811 12:09:58.590847 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:58.590862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:58.590912 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:09:58.593340 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:58.593367 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:58.593832 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:58.594259 (+ 427us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:58.595529 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:58.597583 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:58.598876 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:58.599920 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:58.600113 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:58.603976 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:58.603987 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:58.604010 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:58.604065 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:58.607951 (+ 3886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:58.608592 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:58.608601 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:58.608609 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:58.608831 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:58.609134 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:58.609139 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:58.609142 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:58.609878 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:58.610172 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:58.610462 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 12:09:58.610903 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":350,"threads_started":1}
I20260811 12:09:58.612159 17465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } }
I20260811 12:09:58.612597 17465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:09:58.612699 17465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 887d1b9d93d242f982cb0516898e5849, State: Initialized, Role: FOLLOWER
I20260811 12:09:58.613272 17465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } }
I20260811 12:09:58.613516 17465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:09:58.613657 17465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:09:58.613832 17465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:09:58.616286 17465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } }
I20260811 12:09:58.616837 17465 leader_election.cc:302] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 887d1b9d93d242f982cb0516898e5849; no voters:
I20260811 12:09:58.617401 17465 leader_election.cc:288] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:09:58.617515 17474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:09:58.618451 17474 raft_consensus.cc:686] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [term 1 LEADER]: Becoming Leader. State: Replica: 887d1b9d93d242f982cb0516898e5849, State: Running, Role: LEADER
I20260811 12:09:58.618917 17474 consensus_queue.cc:237] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } }
I20260811 12:09:58.619208 17465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:09:58.621512 17475 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "887d1b9d93d242f982cb0516898e5849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } } }
I20260811 12:09:58.621841 17475 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:58.622262 17478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 887d1b9d93d242f982cb0516898e5849. Latest consensus state: current_term: 1 leader_uuid: "887d1b9d93d242f982cb0516898e5849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "887d1b9d93d242f982cb0516898e5849" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36375 } } }
I20260811 12:09:58.622538 17478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849 [sys.catalog]: This master's current role is: LEADER
I20260811 12:09:58.624087 17486 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:09:58.629303 17486 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:09:58.638103 17486 catalog_manager.cc:1347] Generated new cluster ID: 6d9ef47db85a44c6b2b630de5a225f77
I20260811 12:09:58.638195 17486 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:09:58.645713 17486 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:09:58.646891 17486 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:09:58.657558 17486 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Generated new TSK 0
I20260811 12:09:58.658329 17486 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
I20260811 12:09:58.729533 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36375
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:59.065230 17513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:59.065505 17513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:59.065613 17513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:59.072846 17513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:59.072953 17513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:59.073043 17513 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:59.073127 17513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:09:59.082436 17513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:59.083715 17513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:59.084980 17513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:59.098026 17519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:59.097954 17521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:59.097988 17526 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:59.098990 17513 server_base.cc:1034] running on GCE node
I20260811 12:09:59.099614 17513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:59.100555 17513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:59.101771 17513 hybrid_clock.cc:648] HybridClock initialized: now 1786450199101706 us; error 39 us; skew 500 ppm
I20260811 12:09:59.102062 17513 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:59 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:59.105129 17513 init.cc:377] Logged in from keytab as oryx/127.4.48.193@KRBTEST.COM (short username oryx)
I20260811 12:09:59.106555 17513 webserver.cc:460] Webserver started at http://127.4.48.193:38637/ using document root <none> and password file <none>
I20260811 12:09:59.107205 17513 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:59.107306 17513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:59.107633 17513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:59.110551 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "30da138ee19744faa9139964eae721d7"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "b2c01a32ca631448f5c636e55d02974e"
server_key_iv: "03935d2eda7ed36e07ebc717bfbb7dde"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.111536 17513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "30da138ee19744faa9139964eae721d7"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "b2c01a32ca631448f5c636e55d02974e"
server_key_iv: "03935d2eda7ed36e07ebc717bfbb7dde"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.116700 17513 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:09:59.120007 17536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:59.121241 17513 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:09:59.121407 17513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "30da138ee19744faa9139964eae721d7"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "b2c01a32ca631448f5c636e55d02974e"
server_key_iv: "03935d2eda7ed36e07ebc717bfbb7dde"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.121600 17513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:59.134076 17513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:59.138234 17513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:59.138496 17513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:59.141237 17513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:59.141328 17513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:59.141477 17513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:59.141592 17513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 11 12:09:59 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:59.164644 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.141727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37675 (local address 127.4.48.254:36375)
0811 12:09:59.142572 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:59.142580 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:59.143612 (+ 1032us) server_negotiation.cc:407] Connection header received
0811 12:09:59.145979 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:59.145985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:59.146083 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:59.146196 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:59.147981 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.149161 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.151440 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.152016 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.152148 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.157307 (+ 5159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:59.157314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:59.157319 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:59.157395 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:59.160431 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.161606 (+ 1175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.161611 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.161615 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.161778 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.162435 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.162439 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.162442 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.162842 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:59.162989 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:59.164197 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0811 12:09:59.164392 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":567,"threads_started":1}
I20260811 12:09:59.166366 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.142455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.193:37675)
0811 12:09:59.143437 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:59.143491 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:59.145593 (+ 2102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:59.146549 (+ 956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:59.146557 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:59.147071 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:59.147757 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.147798 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.149362 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.149380 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.151284 (+ 1904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.151301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.152299 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.152307 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.152668 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.153447 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:59.153483 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:59.156971 (+ 3488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:59.160574 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.160587 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.160604 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.161420 (+ 816us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.161946 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.161951 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.161954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.162268 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.163070 (+ 802us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:59.163078 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:59.163639 (+ 561us) client_negotiation.cc:769] Sending connection context
0811 12:09:59.164169 (+ 530us) client_negotiation.cc:240] Negotiation successful
0811 12:09:59.164766 (+ 597us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":425,"threads_started":1}
I20260811 12:09:59.189208 17513 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35473
I20260811 12:09:59.189280 17760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35473 every 8 connection(s)
I20260811 12:09:59.190795 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.172547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:59137 (local address 127.4.48.254:36375)
0811 12:09:59.172721 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:59.172727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:59.176761 (+ 4034us) server_negotiation.cc:407] Connection header received
0811 12:09:59.176958 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:59.176963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:59.177055 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:59.177138 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:59.178555 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.179268 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.181480 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.181864 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.181969 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.184690 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:59.184697 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:59.184701 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:59.184741 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:59.187980 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.188897 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.188900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.188903 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.189068 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.189548 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.189552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.189554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.189825 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:59.189938 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:59.190390 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 12:09:59.190563 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:09:59.190801 17513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:09:59.191071 17683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.172627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.193:59137)
0811 12:09:59.176646 (+ 4019us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:59.176674 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:59.176832 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:59.177197 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:59.177202 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:59.177890 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:59.178392 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.178408 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.179430 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.179436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.181329 (+ 1893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.181343 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.182078 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.182085 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.182309 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.183094 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:59.183123 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:59.184457 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:59.188147 (+ 3690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.188151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.188155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.188758 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.189198 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.189203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.189206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.189356 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.189941 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:59.189946 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:59.190148 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 12:09:59.190305 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 12:09:59.190638 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3839,"thread_start_us":3567,"threads_started":1}
I20260811 12:09:59.192894 17763 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36375
I20260811 12:09:59.193149 17763 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:59.193954 17763 heartbeater.cc:511] Master 127.4.48.254:36375 requested a full tablet report, sending...
I20260811 12:09:59.195019 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 17513
I20260811 12:09:59.195212 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:09:59.196105 4291 external_mini_cluster.cc:1428] Setting key 98ea3018e0493e62dfec1ccf7728bd64
I20260811 12:09:59.196416 17384 ts_manager.cc:194] Registered new tserver with Master: 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473)
I20260811 12:09:59.198294 17384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:37675
WARNING: no policy specified for oryx/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
I20260811 12:09:59.247839 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36375
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:09:59.572867 17770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:09:59.573136 17770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:09:59.573275 17770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:09:59.580305 17770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:09:59.580416 17770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:09:59.580534 17770 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:09:59.580643 17770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:09:59.589936 17770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:09:59.591084 17770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:09:59.592418 17770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:09:59.604679 17786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:59.604713 17780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:09:59.604760 17781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:09:59.605311 17770 server_base.cc:1034] running on GCE node
I20260811 12:09:59.605835 17770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:09:59.606644 17770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:09:59.607833 17770 hybrid_clock.cc:648] HybridClock initialized: now 1786450199607768 us; error 43 us; skew 500 ppm
I20260811 12:09:59.608104 17770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:09:59 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:09:59.611073 17770 init.cc:377] Logged in from keytab as oryx/127.4.48.194@KRBTEST.COM (short username oryx)
I20260811 12:09:59.612504 17770 webserver.cc:460] Webserver started at http://127.4.48.194:44095/ using document root <none> and password file <none>
I20260811 12:09:59.613153 17770 fs_manager.cc:359] Metadata directory not provided
I20260811 12:09:59.613243 17770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:09:59.613549 17770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:09:59.616339 17770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "aaec43f412df4079a882d5dd1b9c4c6e"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "d22036c82f0415010a67a25b1224d87b"
server_key_iv: "fbc99a42e459ba6567d825e74c60850d"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.617089 17770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "aaec43f412df4079a882d5dd1b9c4c6e"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "d22036c82f0415010a67a25b1224d87b"
server_key_iv: "fbc99a42e459ba6567d825e74c60850d"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.621994 17770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:09:59.625334 17799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:59.626585 17770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:09:59.626719 17770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "aaec43f412df4079a882d5dd1b9c4c6e"
format_stamp: "Formatted at 2026-08-11 12:09:59 on dist-test-slave-nj21"
server_key: "d22036c82f0415010a67a25b1224d87b"
server_key_iv: "fbc99a42e459ba6567d825e74c60850d"
server_key_version: "encryptionkey@0"
I20260811 12:09:59.626917 17770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:09:59.647307 17770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:09:59.651149 17770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:09:59.651396 17770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:09:59.653971 17770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:09:59.654048 17770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:09:59.654186 17770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:09:59.654239 17770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:09:59 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450199, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:09:59.677173 17814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.655097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.194:52383)
0811 12:09:59.656000 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:59.656058 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:59.658022 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:59.658892 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:59.658902 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:59.659560 (+ 658us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:59.660400 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.660428 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.661844 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.661849 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.663875 (+ 2026us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.663887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.664845 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.664852 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.665063 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.665685 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:59.665711 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:59.668169 (+ 2458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:59.671993 (+ 3824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.672001 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.672015 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.672456 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.672852 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.672857 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.672860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.673186 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.674004 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:59.674014 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:59.674512 (+ 498us) client_negotiation.cc:769] Sending connection context
0811 12:09:59.674847 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 12:09:59.675396 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":379,"threads_started":1}
I20260811 12:09:59.677883 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.654649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:52383 (local address 127.4.48.254:36375)
0811 12:09:59.654853 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:59.654859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:59.656198 (+ 1339us) server_negotiation.cc:407] Connection header received
0811 12:09:59.658385 (+ 2187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:59.658392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:59.658489 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:59.658605 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:59.660652 (+ 2047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.661653 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.664014 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.664596 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.664711 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.668393 (+ 3682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:59.668398 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:59.668401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:59.668434 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:59.671846 (+ 3412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.672580 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.672585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.672589 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.672721 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.673368 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.673385 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.673389 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.673760 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:59.673919 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:59.677076 (+ 3157us) server_negotiation.cc:299] Negotiation successful
0811 12:09:59.677410 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:09:59.697775 17929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.685199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.194:33355)
0811 12:09:59.685941 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0811 12:09:59.685956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:09:59.686076 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:09:59.686359 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:09:59.686363 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:09:59.686795 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:09:59.687064 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.687072 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.688063 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.688068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.689173 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:09:59.689186 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.689892 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.689898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:09:59.690051 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.690610 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:09:59.690639 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:09:59.691887 (+ 1248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:09:59.694773 (+ 2886us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.694779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.694783 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.695325 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.695732 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:09:59.695737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:09:59.695740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:09:59.695925 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:09:59.696744 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:09:59.696753 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:09:59.696990 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 12:09:59.697165 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 12:09:59.697472 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":496,"threads_started":1}
I20260811 12:09:59.698062 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:09:59.685095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:33355 (local address 127.4.48.254:36375)
0811 12:09:59.685264 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:09:59.685270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:09:59.686035 (+ 765us) server_negotiation.cc:407] Connection header received
0811 12:09:59.686180 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:09:59.686185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:09:59.686246 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:09:59.686323 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:09:59.687193 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.687934 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.689303 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:09:59.689717 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:09:59.689833 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:09:59.692066 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:09:59.692070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:09:59.692073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:09:59.692103 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:09:59.694660 (+ 2557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.695456 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.695461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.695465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.695605 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:09:59.696102 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:09:59.696107 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:09:59.696110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:09:59.696517 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:09:59.696641 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:09:59.697287 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 12:09:59.697534 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:09:59.708545 17770 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:44467
I20260811 12:09:59.708585 18010 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:44467 every 8 connection(s)
I20260811 12:09:59.709686 17770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:09:59.711459 18015 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36375
I20260811 12:09:59.711660 18015 heartbeater.cc:464] Registering TS with master...
I20260811 12:09:59.712291 18015 heartbeater.cc:511] Master 127.4.48.254:36375 requested a full tablet report, sending...
I20260811 12:09:59.713584 17384 ts_manager.cc:194] Registered new tserver with Master: aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467)
I20260811 12:09:59.714407 17384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:52383
I20260811 12:09:59.719133 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 17770
I20260811 12:09:59.719277 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:09:59.719740 4291 external_mini_cluster.cc:1428] Setting key f80a1ce2052e3f2b204d8871380ef251
WARNING: no policy specified for oryx/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
I20260811 12:09:59.769940 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:36375
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:00.088457 18020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:00.088698 18020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:00.088836 18020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:00.096024 18020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:00.096118 18020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:00.096241 18020 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 12:10:00.096347 18020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:00.105732 18020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.4.48.254:36375
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:00.106926 18020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:00.108220 18020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:00.120609 18025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:00.120653 18030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:00.120570 18028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:00.126777 18020 server_base.cc:1034] running on GCE node
I20260811 12:10:00.127336 18020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:00.128275 18020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:00.129501 18020 hybrid_clock.cc:648] HybridClock initialized: now 1786450200129415 us; error 46 us; skew 500 ppm
I20260811 12:10:00.129805 18020 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450200, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:00.132962 18020 init.cc:377] Logged in from keytab as oryx/127.4.48.195@KRBTEST.COM (short username oryx)
I20260811 12:10:00.134438 18020 webserver.cc:460] Webserver started at http://127.4.48.195:43451/ using document root <none> and password file <none>
I20260811 12:10:00.135116 18020 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:00.135215 18020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:00.135646 18020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:00.138551 18020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "2c2e38d08273473780f61c19fbc1a4a3"
format_stamp: "Formatted at 2026-08-11 12:10:00 on dist-test-slave-nj21"
server_key: "a39f9e18342ec4f51bc6197adf9c163e"
server_key_iv: "2348fcc1228e51a3b006fb68aa48736b"
server_key_version: "encryptionkey@0"
I20260811 12:10:00.139295 18020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "2c2e38d08273473780f61c19fbc1a4a3"
format_stamp: "Formatted at 2026-08-11 12:10:00 on dist-test-slave-nj21"
server_key: "a39f9e18342ec4f51bc6197adf9c163e"
server_key_iv: "2348fcc1228e51a3b006fb68aa48736b"
server_key_version: "encryptionkey@0"
I20260811 12:10:00.144518 18020 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 12:10:00.147789 18039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:00.149078 18020 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:10:00.149230 18020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "2c2e38d08273473780f61c19fbc1a4a3"
format_stamp: "Formatted at 2026-08-11 12:10:00 on dist-test-slave-nj21"
server_key: "a39f9e18342ec4f51bc6197adf9c163e"
server_key_iv: "2348fcc1228e51a3b006fb68aa48736b"
server_key_version: "encryptionkey@0"
I20260811 12:10:00.149422 18020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:00.166499 18020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:00.170315 18020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:00.170547 18020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:00.173130 18020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:00.173223 18020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:00.173377 18020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:00.173439 18020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450200, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:10:00.195295 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.173524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:49851 (local address 127.4.48.254:36375)
0811 12:10:00.173750 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.173758 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.175076 (+ 1318us) server_negotiation.cc:407] Connection header received
0811 12:10:00.177098 (+ 2022us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.177106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.177234 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.177426 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:00.179475 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.180499 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.182805 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.183325 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.183458 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.187966 (+ 4508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:00.187972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:00.187977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:00.188022 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:00.191706 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.192780 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.192786 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.192790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.192963 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.193447 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.193452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.193456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.193822 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:00.193986 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.194654 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.194905 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:10:00.195917 18054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.174041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.195:49851)
0811 12:10:00.174886 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.174934 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.176822 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.177670 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.177682 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.178393 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:00.179281 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.179312 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.180679 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.180685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.182609 (+ 1924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.182624 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.183653 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.183664 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.184050 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.184818 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:00.184852 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:00.187649 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:00.191889 (+ 4240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:00.191899 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:00.191917 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:00.192589 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:00.193081 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:00.193086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:00.193089 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:00.193323 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:00.193962 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:00.193967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:00.194259 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.194480 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.194809 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":363,"threads_started":1}
I20260811 12:10:00.201010 17763 heartbeater.cc:503] Master 127.4.48.254:36375 was elected leader, sending a full tablet report...
I20260811 12:10:00.220338 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.206659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:40761 (local address 127.4.48.254:36375)
0811 12:10:00.206839 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.206845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.207349 (+ 504us) server_negotiation.cc:407] Connection header received
0811 12:10:00.207465 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.207471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.207563 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.207691 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:00.208854 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.209793 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.211702 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.212111 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.212218 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.214423 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:00.214428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:00.214431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:00.214464 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:00.217489 (+ 3025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.218299 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.218305 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.218308 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.218443 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.218782 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.218788 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.218791 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.219065 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:00.219210 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.219746 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.219969 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:10:00.220410 18182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.206650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.4.48.195:40761)
0811 12:10:00.207257 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.207281 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.207383 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.207703 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.207708 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.208268 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:00.208711 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.208728 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.209946 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.209952 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.211551 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.211563 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.212331 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.212339 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.212474 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.213142 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:00.213169 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:00.214179 (+ 1010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:00.217649 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:00.217654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:00.217657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:00.218138 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:00.218549 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:00.218554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:00.218558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:00.218682 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:00.219276 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:00.219285 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:00.219480 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.219606 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.219950 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":325,"threads_started":1}
I20260811 12:10:00.224085 18020 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:43483
I20260811 12:10:00.224105 18254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:43483 every 8 connection(s)
I20260811 12:10:00.225327 18020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:00.227229 18255 heartbeater.cc:347] Connected to a master server at 127.4.48.254:36375
I20260811 12:10:00.227438 18255 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:00.228091 18255 heartbeater.cc:511] Master 127.4.48.254:36375 requested a full tablet report, sending...
I20260811 12:10:00.228595 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 18020
I20260811 12:10:00.228745 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:10:00.229214 4291 external_mini_cluster.cc:1428] Setting key 89b5b4321e04eedf31ec3350f5b63c14
I20260811 12:10:00.229411 17384 ts_manager.cc:194] Registered new tserver with Master: 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:00.230387 17384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:49851
I20260811 12:10:00.232087 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:10:00.287506 17551 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:00.277445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36242 (local address 127.4.48.254:36375)
0811 12:10:00.277664 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.277670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.277850 (+ 180us) server_negotiation.cc:407] Connection header received
0811 12:10:00.278052 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.278056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.278121 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.278210 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:00.279527 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.280331 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.281570 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.281981 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.282092 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.287189 (+ 5097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36242: BlockingRecv error: recv got EOF from 127.0.0.1:36242 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450200, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:10:00.302735 17551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.290045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36246 (local address 127.4.48.254:36375)
0811 12:10:00.290199 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.290205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.290520 (+ 315us) server_negotiation.cc:407] Connection header received
0811 12:10:00.290670 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.290676 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.290752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.290831 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:00.291936 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.292662 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.293963 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.294397 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.294505 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.297985 (+ 3480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:00.297990 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:00.297993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:00.298026 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:00.300336 (+ 2310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.300940 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.300946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.300950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.301082 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:00.301423 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:00.301428 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:00.301432 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:00.301685 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:00.301984 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.302217 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.302444 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 12:10:00.306329 17384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:10:00.313081 17384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:00.345480 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.325146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50522 (local address 127.4.48.195:43483)
0811 12:10:00.335949 (+ 10803us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.335958 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.336468 (+ 510us) server_negotiation.cc:407] Connection header received
0811 12:10:00.337140 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.337148 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.337349 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.337526 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:00.338557 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.339741 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.342076 (+ 2335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.343546 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.344009 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.344254 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.344983 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.345141 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10613,"thread_start_us":10788,"threads_started":1}
I20260811 12:10:00.345547 18286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.324751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:43483 (local address 127.0.0.1:50522)
0811 12:10:00.336339 (+ 11588us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.336359 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.336931 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.337806 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.337808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.337828 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:00.338327 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.338345 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.339913 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.339917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.341919 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.341931 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.344400 (+ 2469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.344407 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.344653 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.344770 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.344917 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.345119 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11458,"mutex_wait_us":166,"thread_start_us":11818,"threads_started":1}
I20260811 12:10:00.345899 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.324707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.4.48.193:35473)
0811 12:10:00.336517 (+ 11810us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.336525 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.336555 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:10:00.336653 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.336661 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.336939 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.337135 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:00.338201 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.339796 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.343774 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.344937 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.345070 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.345258 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.345481 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.345637 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11665,"thread_start_us":11515,"threads_started":1}
I20260811 12:10:00.345999 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.326818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:44467 (local address 127.0.0.1:48170)
0811 12:10:00.336840 (+ 10022us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.336858 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.337055 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.337927 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.337931 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.337962 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:00.338478 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.338491 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.340206 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.340215 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.342521 (+ 2306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.342534 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.344431 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.344436 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.344654 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.344770 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.344917 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.345645 (+ 728us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9914,"mutex_wait_us":782,"thread_start_us":6606,"threads_started":1}
I20260811 12:10:00.346166 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.327332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48170 (local address 127.4.48.194:44467)
0811 12:10:00.334734 (+ 7402us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.334743 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.336952 (+ 2209us) server_negotiation.cc:407] Connection header received
0811 12:10:00.337232 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.337239 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.337504 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.337668 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:00.338657 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.340051 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.342681 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.344017 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.344144 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.344306 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.345723 (+ 1417us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.345894 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7210,"thread_start_us":7074,"threads_started":1}
I20260811 12:10:00.346361 18293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.324546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35473 (local address 127.0.0.1:58238)
0811 12:10:00.333881 (+ 9335us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.333933 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.334133 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.337481 (+ 3348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.337492 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.337523 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:00.337983 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.337995 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.339946 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.339951 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.342519 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.342534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.345128 (+ 2594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.345134 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.345289 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.345323 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.345408 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.345535 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8998,"mutex_wait_us":97,"thread_start_us":11929,"threads_started":1}
I20260811 12:10:00.349516 18135 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd (DEFAULT_TABLE table=test-table [id=6161a560a3a8429fbc0c309c1a355d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.349715 17882 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd (DEFAULT_TABLE table=test-table [id=6161a560a3a8429fbc0c309c1a355d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.350783 18135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5973ae6b2a408fb9709eee999c0dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.350313 17627 tablet_service.cc:1467] Processing CreateTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd (DEFAULT_TABLE table=test-table [id=6161a560a3a8429fbc0c309c1a355d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.351073 17882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5973ae6b2a408fb9709eee999c0dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.351563 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f5973ae6b2a408fb9709eee999c0dcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.362006 18295 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Bootstrap starting.
I20260811 12:10:00.364971 18296 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Bootstrap starting.
I20260811 12:10:00.366233 18295 tablet_bootstrap.cc:654] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.367496 18295 log.cc:824] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:00.368335 18296 tablet_bootstrap.cc:654] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.368886 18297 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Bootstrap starting.
I20260811 12:10:00.370148 18296 log.cc:824] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:00.370637 18295 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: No bootstrap required, opened a new log
I20260811 12:10:00.370963 18295 ts_tablet_manager.cc:1397] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:10:00.372164 18297 tablet_bootstrap.cc:654] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.373119 18296 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: No bootstrap required, opened a new log
I20260811 12:10:00.373374 18296 ts_tablet_manager.cc:1397] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:10:00.373345 18297 log.cc:824] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:00.376224 18297 tablet_bootstrap.cc:492] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: No bootstrap required, opened a new log
I20260811 12:10:00.376480 18297 ts_tablet_manager.cc:1397] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:10:00.378309 18295 raft_consensus.cc:348] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.378842 18295 raft_consensus.cc:374] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.379005 18295 raft_consensus.cc:729] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e38d08273473780f61c19fbc1a4a3, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.379622 18295 consensus_queue.cc:260] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.380686 18255 heartbeater.cc:503] Master 127.4.48.254:36375 was elected leader, sending a full tablet report...
I20260811 12:10:00.381234 18295 ts_tablet_manager.cc:1428] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 12:10:00.383121 18296 raft_consensus.cc:348] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.383523 18296 raft_consensus.cc:374] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.383616 18296 raft_consensus.cc:729] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.384234 18296 consensus_queue.cc:260] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.385251 18015 heartbeater.cc:503] Master 127.4.48.254:36375 was elected leader, sending a full tablet report...
I20260811 12:10:00.387029 18296 ts_tablet_manager.cc:1428] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 12:10:00.387840 18297 raft_consensus.cc:348] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.388273 18297 raft_consensus.cc:374] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.388379 18297 raft_consensus.cc:729] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30da138ee19744faa9139964eae721d7, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.388948 18297 consensus_queue.cc:260] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.396520 18297 ts_tablet_manager.cc:1428] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Time spent starting tablet: real 0.020s user 0.018s sys 0.000s
W20260811 12:10:00.443367 17764 tablet.cc:2369] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:10:00.461511 18016 tablet.cc:2369] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:10:00.476881 18257 tablet.cc:2369] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:10:00.565857 18305 raft_consensus.cc:482] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:00.566162 18305 raft_consensus.cc:504] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.567960 18305 leader_election.cc:288] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:00.574792 18288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.568341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:44255 (local address 127.4.48.193:35473)
0811 12:10:00.568569 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.568575 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.569206 (+ 631us) server_negotiation.cc:407] Connection header received
0811 12:10:00.569399 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.569404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.569486 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.569561 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:00.570137 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.570831 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.572746 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.573851 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.574006 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.574077 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.574442 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.574563 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 12:10:00.574818 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.568444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:46157 (local address 127.4.48.195:43483)
0811 12:10:00.568637 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.568643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.569281 (+ 638us) server_negotiation.cc:407] Connection header received
0811 12:10:00.569499 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.569505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.569578 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.569651 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:00.570292 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.570999 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.572925 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.573825 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.573972 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.574050 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.574430 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.574557 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:10:00.574944 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.568195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:43483 (local address 127.4.48.194:46157)
0811 12:10:00.569195 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.569214 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.569344 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.569726 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.569732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.569770 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:00.570137 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.570146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.571133 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.571139 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.572769 (+ 1630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.572782 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.574044 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.574051 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.574253 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.574291 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.574365 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.574484 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":878,"mutex_wait_us":18,"thread_start_us":454,"threads_started":1}
I20260811 12:10:00.575273 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.568156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35473 (local address 127.4.48.194:44255)
0811 12:10:00.569103 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.569120 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.569245 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.569593 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.569598 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.569621 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:00.570000 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.570010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.570955 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.570960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.572577 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.572592 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.574076 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.574083 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.574267 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.574308 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 12:10:00.574396 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:10:00.574516 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":45,"thread_start_us":640,"threads_started":1}
I20260811 12:10:00.576005 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5973ae6b2a408fb9709eee999c0dcd" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3" is_pre_election: true
I20260811 12:10:00.576177 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5973ae6b2a408fb9709eee999c0dcd" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7" is_pre_election: true
I20260811 12:10:00.576514 18167 raft_consensus.cc:2454] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:00.576645 17657 raft_consensus.cc:2454] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:00.577153 17801 leader_election.cc:302] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:00.577523 18305 raft_consensus.cc:2781] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:00.577689 18305 raft_consensus.cc:482] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:00.577792 18305 raft_consensus.cc:3037] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.579926 18305 raft_consensus.cc:504] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.580571 18305 leader_election.cc:288] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Requested vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:00.580951 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5973ae6b2a408fb9709eee999c0dcd" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7"
I20260811 12:10:00.581049 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f5973ae6b2a408fb9709eee999c0dcd" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3"
I20260811 12:10:00.581144 17657 raft_consensus.cc:3037] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.581254 18167 raft_consensus.cc:3037] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.584280 17657 raft_consensus.cc:2454] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:00.584280 18167 raft_consensus.cc:2454] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:00.584637 17801 leader_election.cc:302] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:00.585011 18305 raft_consensus.cc:2781] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:00.585436 18305 raft_consensus.cc:686] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Becoming Leader. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Running, Role: LEADER
I20260811 12:10:00.585938 18305 consensus_queue.cc:237] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } }
I20260811 12:10:00.590746 17383 catalog_manager.cc:5495] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194). New cstate: current_term: 1 leader_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } health_report { overall_health: HEALTHY } } }
I20260811 12:10:00.642092 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.636999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48186 (local address 127.4.48.194:44467)
0811 12:10:00.637185 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.637191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.637273 (+ 82us) server_negotiation.cc:407] Connection header received
0811 12:10:00.637502 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.637506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.637590 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.637713 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:00.638074 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.638763 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.640111 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.640510 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.640617 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.641017 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:00.641522 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:00.641651 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:00.641760 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 12:10:00.641850 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:10:00.649820 18167 raft_consensus.cc:1261] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:00.649852 17657 raft_consensus.cc:1261] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:00.650674 18311 consensus_queue.cc:1035] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:00.651027 18305 consensus_queue.cc:1035] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:00.660326 18314 mvcc.cc:205] Tried to move back new op lower bound from 7317300021852180480 to 7317300021603643392. Current Snapshot: MvccSnapshot[applied={T|T < 7317300021852180480}]
I20260811 12:10:00.660480 18315 mvcc.cc:205] Tried to move back new op lower bound from 7317300021852180480 to 7317300021603643392. Current Snapshot: MvccSnapshot[applied={T|T < 7317300021852180480}]
I20260811 12:10:00.672900 18320 mvcc.cc:205] Tried to move back new op lower bound from 7317300021852180480 to 7317300021603643392. Current Snapshot: MvccSnapshot[applied={T|T < 7317300021852180480}]
I20260811 12:10:00.673959 17383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:00.674242 17383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:00.678352 17383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Sending DeleteTablet for 3 replicas of tablet 8f5973ae6b2a408fb9709eee999c0dcd
I20260811 12:10:00.679350 17627 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:00 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260811 12:10:00.679973 18135 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:00 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260811 12:10:00.681962 18129 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:00 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260811 12:10:00.683138 17882 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f5973ae6b2a408fb9709eee999c0dcd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:00 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:48170
I20260811 12:10:00.684233 17383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:10:00.684892 17383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:00.692459 17879 tablet_service.cc:1467] Processing CreateTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 (DEFAULT_TABLE table=test-table [id=236bfec744894c0b84483cd2fd4d9a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.692605 17625 tablet_service.cc:1467] Processing CreateTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 (DEFAULT_TABLE table=test-table [id=236bfec744894c0b84483cd2fd4d9a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.692584 18129 tablet_service.cc:1467] Processing CreateTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 (DEFAULT_TABLE table=test-table [id=236bfec744894c0b84483cd2fd4d9a63]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:00.693087 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aad9f07c03b4088bf67cd761991ef59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.693133 17625 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aad9f07c03b4088bf67cd761991ef59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.693150 18129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6aad9f07c03b4088bf67cd761991ef59. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:00.699028 18297 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Bootstrap starting.
I20260811 12:10:00.700714 18339 tablet_replica.cc:330] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: stopping tablet replica
I20260811 12:10:00.700937 18297 tablet_bootstrap.cc:654] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.701187 18339 raft_consensus.cc:2229] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:00.701663 18339 raft_consensus.cc:2258] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:00.702256 18296 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Bootstrap starting.
I20260811 12:10:00.703954 18296 tablet_bootstrap.cc:654] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.704228 18340 tablet_replica.cc:330] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: stopping tablet replica
I20260811 12:10:00.704681 18340 raft_consensus.cc:2229] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:10:00.705713 18295 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Bootstrap starting.
I20260811 12:10:00.706280 18297 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: No bootstrap required, opened a new log
I20260811 12:10:00.706435 18297 ts_tablet_manager.cc:1397] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.003s
I20260811 12:10:00.707247 18295 tablet_bootstrap.cc:654] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:00.707561 18297 raft_consensus.cc:348] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.711764 18297 raft_consensus.cc:374] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.711941 18297 raft_consensus.cc:729] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30da138ee19744faa9139964eae721d7, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.712257 18297 consensus_queue.cc:260] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.713070 18297 ts_tablet_manager.cc:1428] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260811 12:10:00.714681 18295 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: No bootstrap required, opened a new log
I20260811 12:10:00.714834 18295 ts_tablet_manager.cc:1397] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 12:10:00.715593 18295 raft_consensus.cc:348] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.715860 18295 raft_consensus.cc:374] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.715976 18295 raft_consensus.cc:729] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e38d08273473780f61c19fbc1a4a3, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.716212 18295 consensus_queue.cc:260] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.717568 18336 tablet_replica.cc:330] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: stopping tablet replica
I20260811 12:10:00.717710 18295 ts_tablet_manager.cc:1428] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:00.718197 18336 raft_consensus.cc:2229] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:00.718719 18336 raft_consensus.cc:2258] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:00.721796 18336 ts_tablet_manager.cc:1905] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:00.723031 18296 tablet_bootstrap.cc:492] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: No bootstrap required, opened a new log
I20260811 12:10:00.723192 18296 ts_tablet_manager.cc:1397] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent bootstrapping tablet: real 0.021s user 0.005s sys 0.004s
I20260811 12:10:00.724128 18296 raft_consensus.cc:348] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.724360 18296 raft_consensus.cc:374] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:00.724478 18296 raft_consensus.cc:729] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Initialized, Role: FOLLOWER
I20260811 12:10:00.724805 18296 consensus_queue.cc:260] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.725431 18296 ts_tablet_manager.cc:1428] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 12:10:00.726511 18340 raft_consensus.cc:2258] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:00.728845 18340 ts_tablet_manager.cc:1905] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:00.729118 18336 ts_tablet_manager.cc:1918] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:00.729327 18336 log.cc:1196] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/8f5973ae6b2a408fb9709eee999c0dcd
I20260811 12:10:00.730046 18336 ts_tablet_manager.cc:1939] T 8f5973ae6b2a408fb9709eee999c0dcd P 30da138ee19744faa9139964eae721d7: Deleting consensus metadata
I20260811 12:10:00.731946 17355 catalog_manager.cc:4841] TS 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473): tablet 8f5973ae6b2a408fb9709eee999c0dcd (table test-table [id=6161a560a3a8429fbc0c309c1a355d1b]) successfully deleted
I20260811 12:10:00.733287 18339 ts_tablet_manager.cc:1905] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:00.739998 18339 ts_tablet_manager.cc:1918] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:00.740166 18339 log.cc:1196] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/8f5973ae6b2a408fb9709eee999c0dcd
I20260811 12:10:00.740679 18339 ts_tablet_manager.cc:1939] T 8f5973ae6b2a408fb9709eee999c0dcd P 2c2e38d08273473780f61c19fbc1a4a3: Deleting consensus metadata
I20260811 12:10:00.740679 18340 ts_tablet_manager.cc:1918] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:00.740927 18340 log.cc:1196] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/8f5973ae6b2a408fb9709eee999c0dcd
I20260811 12:10:00.741516 18340 ts_tablet_manager.cc:1939] T 8f5973ae6b2a408fb9709eee999c0dcd P aaec43f412df4079a882d5dd1b9c4c6e: Deleting consensus metadata
I20260811 12:10:00.742408 17350 catalog_manager.cc:4841] TS 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483): tablet 8f5973ae6b2a408fb9709eee999c0dcd (table test-table [id=6161a560a3a8429fbc0c309c1a355d1b]) successfully deleted
W20260811 12:10:00.743445 17350 catalog_manager.cc:4808] TS 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483): delete failed for tablet 8f5973ae6b2a408fb9709eee999c0dcd because the tablet was not found. No further retry: Not found: Tablet not found: 8f5973ae6b2a408fb9709eee999c0dcd
I20260811 12:10:00.744199 17352 catalog_manager.cc:4841] TS aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467): tablet 8f5973ae6b2a408fb9709eee999c0dcd (table test-table [id=6161a560a3a8429fbc0c309c1a355d1b]) successfully deleted
I20260811 12:10:00.906507 18311 raft_consensus.cc:482] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:00.906737 18311 raft_consensus.cc:504] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.907505 18311 leader_election.cc:288] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:00.908135 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aad9f07c03b4088bf67cd761991ef59" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7" is_pre_election: true
I20260811 12:10:00.908135 18165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aad9f07c03b4088bf67cd761991ef59" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3" is_pre_election: true
I20260811 12:10:00.908396 18165 raft_consensus.cc:2454] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:00.908396 17657 raft_consensus.cc:2454] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:00.908857 17801 leader_election.cc:302] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:00.909143 18311 raft_consensus.cc:2781] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:00.909235 18311 raft_consensus.cc:482] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:00.909354 18311 raft_consensus.cc:3037] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.911517 18311 raft_consensus.cc:504] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.912092 18311 leader_election.cc:288] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Requested vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:00.912488 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aad9f07c03b4088bf67cd761991ef59" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7"
I20260811 12:10:00.912562 18165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6aad9f07c03b4088bf67cd761991ef59" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3"
I20260811 12:10:00.912705 17657 raft_consensus.cc:3037] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.912734 18165 raft_consensus.cc:3037] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:00.914655 18165 raft_consensus.cc:2454] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:00.914992 17801 leader_election.cc:302] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:00.915217 18311 raft_consensus.cc:2781] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:00.915408 18311 raft_consensus.cc:686] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Becoming Leader. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Running, Role: LEADER
I20260811 12:10:00.915642 17657 raft_consensus.cc:2454] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:00.915922 18311 consensus_queue.cc:237] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:00.918588 17383 catalog_manager.cc:5495] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194). New cstate: current_term: 1 leader_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } health_report { overall_health: UNKNOWN } } }
Aug 11 12:10:00 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450198, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.254@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:10:01.007261 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.981934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:36375 (local address 127.0.0.1:36256)
0811 12:10:00.984230 (+ 2296us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:00.984259 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:00.984427 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:00.984843 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:00.984852 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:00.985831 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:00.986077 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.986091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.987078 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.987084 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.988577 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:00.988593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.989420 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.989427 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:00.989617 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:00.999312 (+ 9695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:00.999344 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:01.002080 (+ 2736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:01.004864 (+ 2784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:01.004876 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:01.004885 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:01.005279 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:01.005622 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:01.005627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:01.005631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:01.005762 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:01.006263 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:01.006271 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:01.006435 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.006583 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.006847 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2104,"thread_start_us":1930,"threads_started":1}
I20260811 12:10:01.007330 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:00.981953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36256 (local address 127.4.48.254:36375)
0811 12:10:00.984518 (+ 2565us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:00.984523 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:00.984543 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:10:00.984587 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:00.984591 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:00.984655 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:00.984738 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:00.986218 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.986940 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.988718 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:00.989224 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:00.989332 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.002275 (+ 12943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:01.002280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:01.002283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:01.002314 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:01.004736 (+ 2422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:01.005398 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:01.005402 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:01.005404 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:01.005524 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:01.005875 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:01.005878 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:01.005880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:01.006139 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:01.006276 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.006697 (+ 421us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.006980 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":96,"server-negotiator.queue_time_us":2463,"thread_start_us":2323,"threads_started":1}
I20260811 12:10:01.012126 17383 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:36256:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 12:10:01.013142 17383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:01.019661 17627 tablet_service.cc:1467] Processing CreateTablet for tablet 854eaf4b881f4838bf89ecb765addf67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c0d8b01bfcf4692b7c9ce9073011a67]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:10:01.020275 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854eaf4b881f4838bf89ecb765addf67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.020809 17879 tablet_service.cc:1467] Processing CreateTablet for tablet 854eaf4b881f4838bf89ecb765addf67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c0d8b01bfcf4692b7c9ce9073011a67]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:10:01.021759 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854eaf4b881f4838bf89ecb765addf67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.026247 18297 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: Bootstrap starting.
I20260811 12:10:01.027869 18297 tablet_bootstrap.cc:654] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.029117 18129 tablet_service.cc:1467] Processing CreateTablet for tablet 854eaf4b881f4838bf89ecb765addf67 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=3c0d8b01bfcf4692b7c9ce9073011a67]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 12:10:01.029686 18129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 854eaf4b881f4838bf89ecb765addf67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.030377 18296 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: Bootstrap starting.
I20260811 12:10:01.032224 18296 tablet_bootstrap.cc:654] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.037057 18295 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: Bootstrap starting.
I20260811 12:10:01.037865 18296 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: No bootstrap required, opened a new log
I20260811 12:10:01.037957 18297 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: No bootstrap required, opened a new log
I20260811 12:10:01.038048 18296 ts_tablet_manager.cc:1397] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:10:01.038120 18297 ts_tablet_manager.cc:1397] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260811 12:10:01.038663 18295 tablet_bootstrap.cc:654] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.038827 18296 raft_consensus.cc:348] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.039079 18296 raft_consensus.cc:374] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.039165 18296 raft_consensus.cc:729] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.039616 18296 consensus_queue.cc:260] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.039839 18297 raft_consensus.cc:348] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.040081 18297 raft_consensus.cc:374] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.040149 18297 raft_consensus.cc:729] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30da138ee19744faa9139964eae721d7, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.040222 18311 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.040517 18311 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.040432 18297 consensus_queue.cc:260] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.041033 18297 ts_tablet_manager.cc:1428] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:01.040921 18306 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.041307 18306 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.043946 18296 ts_tablet_manager.cc:1428] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260811 12:10:01.044119 18295 tablet_bootstrap.cc:492] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: No bootstrap required, opened a new log
I20260811 12:10:01.044335 18295 ts_tablet_manager.cc:1397] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260811 12:10:01.045089 18295 raft_consensus.cc:348] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.045318 18295 raft_consensus.cc:374] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.045405 18295 raft_consensus.cc:729] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e38d08273473780f61c19fbc1a4a3, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.045666 18295 consensus_queue.cc:260] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.046109 18304 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.046438 18304 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.046864 18295 ts_tablet_manager.cc:1428] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:10:01.048691 18306 raft_consensus.cc:482] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:01.048861 18306 raft_consensus.cc:504] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.050686 18306 leader_election.cc:288] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.059049 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.051327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:47165 (local address 127.4.48.195:43483)
0811 12:10:01.051500 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.051506 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.051999 (+ 493us) server_negotiation.cc:407] Connection header received
0811 12:10:01.052215 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.052222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.052311 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.052384 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:01.053223 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.054397 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.056955 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.057916 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.058036 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.058114 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.058651 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.058785 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:10:01.059093 18372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.050987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:43483 (local address 127.4.48.193:47165)
0811 12:10:01.051890 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:01.051910 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:01.052046 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:01.052606 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:01.052611 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:01.052638 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:01.053030 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.053042 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.054534 (+ 1492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.054539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.056782 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.056796 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.058297 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.058303 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.058479 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.058510 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.058602 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.058731 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":788,"thread_start_us":622,"threads_started":1}
I20260811 12:10:01.059826 18165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854eaf4b881f4838bf89ecb765addf67" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3" is_pre_election: true
I20260811 12:10:01.060066 18165 raft_consensus.cc:2454] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 0.
I20260811 12:10:01.060540 17537 leader_election.cc:302] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, 30da138ee19744faa9139964eae721d7; no voters:
I20260811 12:10:01.060956 18306 raft_consensus.cc:2781] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:01.061086 18306 raft_consensus.cc:482] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:01.061187 18306 raft_consensus.cc:3037] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.064188 18306 raft_consensus.cc:504] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.064378 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.051449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:33187 (local address 127.4.48.194:44467)
0811 12:10:01.051608 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.051613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.056308 (+ 4695us) server_negotiation.cc:407] Connection header received
0811 12:10:01.056578 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.056583 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.056656 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.056715 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:01.057489 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.058601 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.061020 (+ 2419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.062283 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.062416 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.062487 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.063962 (+ 1475us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.064089 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:10:01.065017 18306 leader_election.cc:288] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 election: Requested vote from peers aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.065083 18374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.050994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:44467 (local address 127.4.48.193:33187)
0811 12:10:01.056203 (+ 5209us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:01.056231 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:01.056414 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:01.056917 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:01.056921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:01.056947 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:01.057331 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.057341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.058850 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.058855 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.060816 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.060830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.063551 (+ 2721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.063557 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.063767 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.063798 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.064584 (+ 786us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.064717 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5059,"mutex_wait_us":3,"thread_start_us":4851,"threads_started":1}
I20260811 12:10:01.065368 18165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854eaf4b881f4838bf89ecb765addf67" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3"
I20260811 12:10:01.065584 18165 raft_consensus.cc:3037] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.068329 18165 raft_consensus.cc:2454] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 1.
I20260811 12:10:01.068692 17537 leader_election.cc:302] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, 30da138ee19744faa9139964eae721d7; no voters:
I20260811 12:10:01.069017 18306 raft_consensus.cc:2781] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:01.069494 18306 raft_consensus.cc:686] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [term 1 LEADER]: Becoming Leader. State: Replica: 30da138ee19744faa9139964eae721d7, State: Running, Role: LEADER
I20260811 12:10:01.069393 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854eaf4b881f4838bf89ecb765addf67" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" is_pre_election: true
I20260811 12:10:01.069886 17930 raft_consensus.cc:2454] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 0.
I20260811 12:10:01.070111 18306 consensus_queue.cc:237] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.070475 17928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "854eaf4b881f4838bf89ecb765addf67" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaec43f412df4079a882d5dd1b9c4c6e"
I20260811 12:10:01.070727 17928 raft_consensus.cc:3037] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.073158 18375 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: TxnStatusTablet state changed. Reason: New leader 30da138ee19744faa9139964eae721d7. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.073489 18375 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:10:01.073671 17928 raft_consensus.cc:2454] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 1.
I20260811 12:10:01.073653 17384 catalog_manager.cc:5495] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30da138ee19744faa9139964eae721d7 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } health_report { overall_health: UNKNOWN } } }
I20260811 12:10:01.081140 18378 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 12:10:01.081373 18378 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 12:10:01.099205 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.089296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35473 (local address 127.0.0.1:58244)
0811 12:10:01.089674 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:01.089696 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:01.089824 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:01.092578 (+ 2754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:01.092583 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:01.092597 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:10:01.092799 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.092811 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.094405 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.094411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.096034 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.096050 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.097100 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.097105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.097312 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.097387 (+ 75us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.098501 (+ 1114us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.098525 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.098614 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.098917 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223}
I20260811 12:10:01.099210 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.089274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.4.48.193:35473)
0811 12:10:01.092122 (+ 2848us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.092128 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.092152 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:10:01.092243 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.092249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.092364 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.092523 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:01.092972 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.094250 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.096181 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.096909 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.097032 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.097643 (+ 611us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.098384 (+ 741us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.098524 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.098664 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.098805 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2701,"thread_start_us":2590,"threads_started":1}
I20260811 12:10:01.103612 18165 raft_consensus.cc:1261] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Refusing update from remote peer 30da138ee19744faa9139964eae721d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.104166 17928 raft_consensus.cc:1261] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Refusing update from remote peer 30da138ee19744faa9139964eae721d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.105005 18306 consensus_queue.cc:1035] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.105391 18375 consensus_queue.cc:1035] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.114457 18311 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: TxnStatusTablet state changed. Reason: New leader 30da138ee19744faa9139964eae721d7. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.114709 18311 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.116887 18306 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.117163 18306 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:10:01.120162 18311 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.120427 18311 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P aaec43f412df4079a882d5dd1b9c4c6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.122154 18304 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: TxnStatusTablet state changed. Reason: New leader 30da138ee19744faa9139964eae721d7. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.122397 18304 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.125229 18375 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.125500 18375 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 30da138ee19744faa9139964eae721d7: This TxnStatusTablet replica's current role is: LEADER
I20260811 12:10:01.128952 18304 tablet_replica.cc:439] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } } }
I20260811 12:10:01.129186 18304 tablet_replica.cc:442] T 854eaf4b881f4838bf89ecb765addf67 P 2c2e38d08273473780f61c19fbc1a4a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 12:10:01.141875 18387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.135760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35473 (local address 127.4.48.194:46951)
0811 12:10:01.136401 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:01.136416 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:01.136536 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:01.136872 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:01.136877 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:01.136890 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:10:01.137229 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.137244 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.138672 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.138676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.139801 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.139815 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.140803 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.140808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.140935 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.140989 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.141399 (+ 410us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.141408 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.141468 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.141568 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":323,"threads_started":1}
I20260811 12:10:01.141906 18381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.135765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:46951 (local address 127.4.48.193:35473)
0811 12:10:01.135948 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.135954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.136504 (+ 550us) server_negotiation.cc:407] Connection header received
0811 12:10:01.136636 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.136642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.136727 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.136824 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:01.137418 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.138529 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.139933 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.140636 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.140763 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.141147 (+ 384us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.141289 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.141400 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.141504 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.141589 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:10:01.152171 18165 raft_consensus.cc:1261] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.152304 17657 raft_consensus.cc:1261] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.152805 18333 consensus_queue.cc:1035] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.153213 18311 consensus_queue.cc:1035] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.188041 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.181420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56755 (local address 127.4.48.194:44467)
0811 12:10:01.181577 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.181582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.182270 (+ 688us) server_negotiation.cc:407] Connection header received
0811 12:10:01.182508 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.182512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.182571 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.182640 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:01.183412 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.184516 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.186082 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.186716 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.186837 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.187331 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.187467 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.187552 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.187707 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.187782 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 12:10:01.188401 18394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.181498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:44467 (local address 127.4.48.193:56755)
0811 12:10:01.182166 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:01.182185 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:01.182360 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:01.182676 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:01.182680 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:01.182693 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:10:01.183237 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.183251 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.184637 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.184641 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.185930 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:01.185945 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.186905 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.186912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:01.187088 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.187154 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.187574 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.187586 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 12:10:01.187659 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:10:01.187828 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":295,"threads_started":1}
I20260811 12:10:01.235417 17384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:01.235630 17384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36246:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:01.238380 17384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Sending DeleteTablet for 3 replicas of tablet 6aad9f07c03b4088bf67cd761991ef59
I20260811 12:10:01.239022 17627 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260811 12:10:01.239285 18129 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260811 12:10:01.239689 18396 tablet_replica.cc:330] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: stopping tablet replica
I20260811 12:10:01.239810 18339 tablet_replica.cc:330] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: stopping tablet replica
I20260811 12:10:01.239888 18396 raft_consensus.cc:2229] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:01.239998 18339 raft_consensus.cc:2229] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:01.240127 18396 raft_consensus.cc:2258] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.240164 18339 raft_consensus.cc:2258] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.239584 17879 tablet_service.cc:1514] Processing DeleteTablet for tablet 6aad9f07c03b4088bf67cd761991ef59 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:48170
I20260811 12:10:01.240710 18340 tablet_replica.cc:330] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: stopping tablet replica
I20260811 12:10:01.240898 18340 raft_consensus.cc:2229] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:10:01.241168 18396 ts_tablet_manager.cc:1905] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.241183 18340 raft_consensus.cc:2258] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.242209 18339 ts_tablet_manager.cc:1905] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.244405 18340 ts_tablet_manager.cc:1905] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.247814 18396 ts_tablet_manager.cc:1918] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:01.247948 18396 log.cc:1196] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/6aad9f07c03b4088bf67cd761991ef59
I20260811 12:10:01.248373 18396 ts_tablet_manager.cc:1939] T 6aad9f07c03b4088bf67cd761991ef59 P 30da138ee19744faa9139964eae721d7: Deleting consensus metadata
I20260811 12:10:01.249332 17355 catalog_manager.cc:4841] TS 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473): tablet 6aad9f07c03b4088bf67cd761991ef59 (table test-table [id=236bfec744894c0b84483cd2fd4d9a63]) successfully deleted
I20260811 12:10:01.250953 18340 ts_tablet_manager.cc:1918] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:01.251103 18340 log.cc:1196] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/6aad9f07c03b4088bf67cd761991ef59
I20260811 12:10:01.251506 18340 ts_tablet_manager.cc:1939] T 6aad9f07c03b4088bf67cd761991ef59 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting consensus metadata
I20260811 12:10:01.257032 18339 ts_tablet_manager.cc:1918] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:01.257193 18339 log.cc:1196] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/6aad9f07c03b4088bf67cd761991ef59
I20260811 12:10:01.257602 18339 ts_tablet_manager.cc:1939] T 6aad9f07c03b4088bf67cd761991ef59 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting consensus metadata
I20260811 12:10:01.258584 17350 catalog_manager.cc:4841] TS 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483): tablet 6aad9f07c03b4088bf67cd761991ef59 (table test-table [id=236bfec744894c0b84483cd2fd4d9a63]) successfully deleted
I20260811 12:10:01.259255 17352 catalog_manager.cc:4841] TS aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467): tablet 6aad9f07c03b4088bf67cd761991ef59 (table test-table [id=236bfec744894c0b84483cd2fd4d9a63]) successfully deleted
Aug 11 12:10:01 dist-test-slave-nj21 krb5kdc[17304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 12:10:01 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:01 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:10:01.307762 18360 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:01.296446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.4.48.254:36375)
0811 12:10:01.296616 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.296622 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.297011 (+ 389us) server_negotiation.cc:407] Connection header received
0811 12:10:01.297212 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.297219 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.297305 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.297399 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:01.299149 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.299962 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.301773 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.302204 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.302297 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.307480 (+ 5183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36262: BlockingRecv error: recv got EOF from 127.0.0.1:36262 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 12:10:01 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM
I20260811 12:10:01.324994 18360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.310566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36264 (local address 127.4.48.254:36375)
0811 12:10:01.310747 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.310752 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.311035 (+ 283us) server_negotiation.cc:407] Connection header received
0811 12:10:01.311202 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.311205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.311264 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.311331 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:01.312728 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.313425 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.315182 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.315570 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.315660 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.319991 (+ 4331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:01.319996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:01.319999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:01.320032 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:01.322515 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:01.323195 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:01.323198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:01.323200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:01.323317 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:01.323706 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:01.323710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:01.323713 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:01.323996 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:01.324187 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.324528 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.324687 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:10:01.328477 17384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:10:01.329079 17384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:01.335716 17627 tablet_service.cc:1467] Processing CreateTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe (DEFAULT_TABLE table=test-table [id=d549d75657344a018327913ac3c55752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.335954 18129 tablet_service.cc:1467] Processing CreateTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe (DEFAULT_TABLE table=test-table [id=d549d75657344a018327913ac3c55752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.335999 17879 tablet_service.cc:1467] Processing CreateTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe (DEFAULT_TABLE table=test-table [id=d549d75657344a018327913ac3c55752]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.336213 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b972c66f09ab43be8e8d9d12c0730dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.336344 18129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b972c66f09ab43be8e8d9d12c0730dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.336429 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b972c66f09ab43be8e8d9d12c0730dfe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.342581 18295 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Bootstrap starting.
I20260811 12:10:01.342767 18296 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Bootstrap starting.
I20260811 12:10:01.344336 18296 tablet_bootstrap.cc:654] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.344343 18295 tablet_bootstrap.cc:654] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.345348 18297 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Bootstrap starting.
I20260811 12:10:01.346149 18295 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: No bootstrap required, opened a new log
I20260811 12:10:01.346278 18295 ts_tablet_manager.cc:1397] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 12:10:01.346896 18297 tablet_bootstrap.cc:654] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.346838 18295 raft_consensus.cc:348] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.347033 18295 raft_consensus.cc:374] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.347119 18295 raft_consensus.cc:729] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e38d08273473780f61c19fbc1a4a3, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.347325 18295 consensus_queue.cc:260] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.348532 18295 ts_tablet_manager.cc:1428] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 12:10:01.350718 18296 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: No bootstrap required, opened a new log
I20260811 12:10:01.350920 18296 ts_tablet_manager.cc:1397] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260811 12:10:01.351717 18296 raft_consensus.cc:348] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.351928 18296 raft_consensus.cc:374] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.352026 18296 raft_consensus.cc:729] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.352232 18296 consensus_queue.cc:260] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.352784 18296 ts_tablet_manager.cc:1428] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 12:10:01.355753 18297 tablet_bootstrap.cc:492] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: No bootstrap required, opened a new log
I20260811 12:10:01.355895 18297 ts_tablet_manager.cc:1397] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260811 12:10:01.356564 18297 raft_consensus.cc:348] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.356803 18297 raft_consensus.cc:374] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.356882 18297 raft_consensus.cc:729] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30da138ee19744faa9139964eae721d7, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.357092 18297 consensus_queue.cc:260] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.357547 18297 ts_tablet_manager.cc:1428] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 12:10:01.494396 18311 raft_consensus.cc:482] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:01.494628 18311 raft_consensus.cc:504] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.495307 18311 leader_election.cc:288] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.495963 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b972c66f09ab43be8e8d9d12c0730dfe" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7" is_pre_election: true
I20260811 12:10:01.496013 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b972c66f09ab43be8e8d9d12c0730dfe" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3" is_pre_election: true
I20260811 12:10:01.496233 17657 raft_consensus.cc:2454] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:01.496232 18167 raft_consensus.cc:2454] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 0.
I20260811 12:10:01.496685 17803 leader_election.cc:302] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30da138ee19744faa9139964eae721d7, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:01.497002 18311 raft_consensus.cc:2781] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:01.497099 18311 raft_consensus.cc:482] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:01.497197 18311 raft_consensus.cc:3037] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.499593 18311 raft_consensus.cc:504] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.500579 17657 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b972c66f09ab43be8e8d9d12c0730dfe" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30da138ee19744faa9139964eae721d7"
I20260811 12:10:01.500604 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b972c66f09ab43be8e8d9d12c0730dfe" candidate_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3"
I20260811 12:10:01.500790 18167 raft_consensus.cc:3037] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.500798 17657 raft_consensus.cc:3037] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.503691 18167 raft_consensus.cc:2454] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:01.503701 17657 raft_consensus.cc:2454] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aaec43f412df4079a882d5dd1b9c4c6e in term 1.
I20260811 12:10:01.503871 18311 leader_election.cc:288] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Requested vote from peers 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.504117 17803 leader_election.cc:302] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2c2e38d08273473780f61c19fbc1a4a3, 30da138ee19744faa9139964eae721d7, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:01.504398 18311 raft_consensus.cc:2781] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:01.504565 18311 raft_consensus.cc:686] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Becoming Leader. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Running, Role: LEADER
I20260811 12:10:01.504823 18311 consensus_queue.cc:237] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.507599 17384 catalog_manager.cc:5495] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e reported cstate change: term changed from 0 to 1, leader changed from <none> to aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194). New cstate: current_term: 1 leader_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } health_report { overall_health: UNKNOWN } } }
I20260811 12:10:01.546738 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:01.541995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48196 (local address 127.4.48.194:44467)
0811 12:10:01.542191 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:01.542198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:01.542292 (+ 94us) server_negotiation.cc:407] Connection header received
0811 12:10:01.542482 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:01.542486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:01.542551 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:01.542623 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:01.542997 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.543838 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.545147 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:01.545564 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:01.545684 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:01.546084 (+ 400us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:01.546194 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:01.546268 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:01.546402 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:10:01.546480 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:10:01.549206 17657 raft_consensus.cc:1261] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.549206 18167 raft_consensus.cc:1261] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Refusing update from remote peer aaec43f412df4079a882d5dd1b9c4c6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:01.549815 18311 consensus_queue.cc:1035] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.550089 18305 consensus_queue.cc:1035] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:01.560374 17384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:01.560602 17384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:01.563328 17384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Sending DeleteTablet for 3 replicas of tablet b972c66f09ab43be8e8d9d12c0730dfe
I20260811 12:10:01.564096 17879 tablet_service.cc:1514] Processing DeleteTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:48170
I20260811 12:10:01.564170 17627 tablet_service.cc:1514] Processing DeleteTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260811 12:10:01.564179 18129 tablet_service.cc:1514] Processing DeleteTablet for tablet b972c66f09ab43be8e8d9d12c0730dfe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:01 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260811 12:10:01.564407 18396 tablet_replica.cc:330] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: stopping tablet replica
I20260811 12:10:01.564400 18340 tablet_replica.cc:330] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: stopping tablet replica
I20260811 12:10:01.564435 18339 tablet_replica.cc:330] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: stopping tablet replica
I20260811 12:10:01.564589 18339 raft_consensus.cc:2229] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:01.564575 18340 raft_consensus.cc:2229] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:10:01.564595 18396 raft_consensus.cc:2229] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:01.564808 18396 raft_consensus.cc:2258] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.564808 18339 raft_consensus.cc:2258] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.564888 18340 raft_consensus.cc:2258] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:01.565764 18340 ts_tablet_manager.cc:1905] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.565932 18339 ts_tablet_manager.cc:1905] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.566250 18396 ts_tablet_manager.cc:1905] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:01.568450 17384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:10:01.569195 17384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:01.571530 18339 ts_tablet_manager.cc:1918] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:01.571703 18339 log.cc:1196] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/b972c66f09ab43be8e8d9d12c0730dfe
I20260811 12:10:01.572113 18339 ts_tablet_manager.cc:1939] T b972c66f09ab43be8e8d9d12c0730dfe P 2c2e38d08273473780f61c19fbc1a4a3: Deleting consensus metadata
I20260811 12:10:01.572417 18396 ts_tablet_manager.cc:1918] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:01.572564 18396 log.cc:1196] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/b972c66f09ab43be8e8d9d12c0730dfe
I20260811 12:10:01.572948 18396 ts_tablet_manager.cc:1939] T b972c66f09ab43be8e8d9d12c0730dfe P 30da138ee19744faa9139964eae721d7: Deleting consensus metadata
I20260811 12:10:01.573089 18340 ts_tablet_manager.cc:1918] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:01.573212 18340 log.cc:1196] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/b972c66f09ab43be8e8d9d12c0730dfe
I20260811 12:10:01.573596 18340 ts_tablet_manager.cc:1939] T b972c66f09ab43be8e8d9d12c0730dfe P aaec43f412df4079a882d5dd1b9c4c6e: Deleting consensus metadata
I20260811 12:10:01.573946 17355 catalog_manager.cc:4841] TS 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473): tablet b972c66f09ab43be8e8d9d12c0730dfe (table test-table [id=d549d75657344a018327913ac3c55752]) successfully deleted
I20260811 12:10:01.574410 17350 catalog_manager.cc:4841] TS 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483): tablet b972c66f09ab43be8e8d9d12c0730dfe (table test-table [id=d549d75657344a018327913ac3c55752]) successfully deleted
I20260811 12:10:01.574704 17352 catalog_manager.cc:4841] TS aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467): tablet b972c66f09ab43be8e8d9d12c0730dfe (table test-table [id=d549d75657344a018327913ac3c55752]) successfully deleted
I20260811 12:10:01.576354 17879 tablet_service.cc:1467] Processing CreateTablet for tablet 2e26eb5597774ab988b11b9631496947 (DEFAULT_TABLE table=test-table [id=4e4d653570b64b8687aabdcdc16ba03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.576351 17627 tablet_service.cc:1467] Processing CreateTablet for tablet 2e26eb5597774ab988b11b9631496947 (DEFAULT_TABLE table=test-table [id=4e4d653570b64b8687aabdcdc16ba03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.576433 18129 tablet_service.cc:1467] Processing CreateTablet for tablet 2e26eb5597774ab988b11b9631496947 (DEFAULT_TABLE table=test-table [id=4e4d653570b64b8687aabdcdc16ba03b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:01.576716 17879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e26eb5597774ab988b11b9631496947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.576727 18129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e26eb5597774ab988b11b9631496947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.576731 17627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2e26eb5597774ab988b11b9631496947. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:01.582459 18296 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Bootstrap starting.
I20260811 12:10:01.583606 18296 tablet_bootstrap.cc:654] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.583917 18295 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Bootstrap starting.
I20260811 12:10:01.585374 18295 tablet_bootstrap.cc:654] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.585493 18296 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: No bootstrap required, opened a new log
I20260811 12:10:01.585645 18296 ts_tablet_manager.cc:1397] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:01.586361 18296 raft_consensus.cc:348] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.586582 18296 raft_consensus.cc:374] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.586650 18296 raft_consensus.cc:729] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aaec43f412df4079a882d5dd1b9c4c6e, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.587857 18296 consensus_queue.cc:260] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.588327 18297 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Bootstrap starting.
I20260811 12:10:01.588862 18296 ts_tablet_manager.cc:1428] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:01.589847 18297 tablet_bootstrap.cc:654] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:01.591724 18295 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: No bootstrap required, opened a new log
I20260811 12:10:01.591876 18295 ts_tablet_manager.cc:1397] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.003s
I20260811 12:10:01.592545 18295 raft_consensus.cc:348] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.592769 18295 raft_consensus.cc:374] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.592849 18295 raft_consensus.cc:729] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c2e38d08273473780f61c19fbc1a4a3, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.593098 18295 consensus_queue.cc:260] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.593607 18295 ts_tablet_manager.cc:1428] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 12:10:01.594476 18297 tablet_bootstrap.cc:492] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: No bootstrap required, opened a new log
I20260811 12:10:01.594666 18297 ts_tablet_manager.cc:1397] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 12:10:01.595381 18297 raft_consensus.cc:348] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.595604 18297 raft_consensus.cc:374] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:01.595715 18297 raft_consensus.cc:729] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30da138ee19744faa9139964eae721d7, State: Initialized, Role: FOLLOWER
I20260811 12:10:01.595953 18297 consensus_queue.cc:260] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.596400 18297 ts_tablet_manager.cc:1428] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 12:10:01.980033 18375 raft_consensus.cc:482] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:01.980237 18375 raft_consensus.cc:504] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.980885 18375 leader_election.cc:288] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.981426 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e26eb5597774ab988b11b9631496947" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" is_pre_election: true
I20260811 12:10:01.981452 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e26eb5597774ab988b11b9631496947" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3" is_pre_election: true
I20260811 12:10:01.981654 17930 raft_consensus.cc:2454] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 0.
I20260811 12:10:01.981664 18167 raft_consensus.cc:2454] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 0.
I20260811 12:10:01.982082 17538 leader_election.cc:302] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30da138ee19744faa9139964eae721d7, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:01.982389 18375 raft_consensus.cc:2781] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:01.982501 18375 raft_consensus.cc:482] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:01.982581 18375 raft_consensus.cc:3037] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.984462 18375 raft_consensus.cc:504] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.984980 18375 leader_election.cc:288] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 election: Requested vote from peers aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467), 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483)
I20260811 12:10:01.985422 17930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e26eb5597774ab988b11b9631496947" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aaec43f412df4079a882d5dd1b9c4c6e"
I20260811 12:10:01.985441 18167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2e26eb5597774ab988b11b9631496947" candidate_uuid: "30da138ee19744faa9139964eae721d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c2e38d08273473780f61c19fbc1a4a3"
I20260811 12:10:01.985595 17930 raft_consensus.cc:3037] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.985595 18167 raft_consensus.cc:3037] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:01.987507 18167 raft_consensus.cc:2454] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 1.
I20260811 12:10:01.987514 17930 raft_consensus.cc:2454] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 30da138ee19744faa9139964eae721d7 in term 1.
I20260811 12:10:01.987882 17538 leader_election.cc:302] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 30da138ee19744faa9139964eae721d7, aaec43f412df4079a882d5dd1b9c4c6e; no voters:
I20260811 12:10:01.988118 18375 raft_consensus.cc:2781] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:01.988512 18375 raft_consensus.cc:686] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 1 LEADER]: Becoming Leader. State: Replica: 30da138ee19744faa9139964eae721d7, State: Running, Role: LEADER
I20260811 12:10:01.988761 18375 consensus_queue.cc:237] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } }
I20260811 12:10:01.991237 17382 catalog_manager.cc:5495] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 30da138ee19744faa9139964eae721d7 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "30da138ee19744faa9139964eae721d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30da138ee19744faa9139964eae721d7" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35473 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 } health_report { overall_health: UNKNOWN } } }
I20260811 12:10:02.050851 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.046416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58246 (local address 127.4.48.193:35473)
0811 12:10:02.046899 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:02.046903 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:02.046918 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:10:02.046967 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:02.046970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:02.047030 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:02.047106 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:02.047463 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.048154 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.049364 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.049749 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.049834 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.050230 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:02.050347 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:02.050410 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:02.050504 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 12:10:02.050561 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":241,"threads_started":1}
I20260811 12:10:02.063236 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.058540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:51401 (local address 127.4.48.193:35473)
0811 12:10:02.058675 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:02.058679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:02.059157 (+ 478us) server_negotiation.cc:407] Connection header received
0811 12:10:02.059307 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:02.059310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:02.059373 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:02.059440 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:02.059867 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.060539 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.061757 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.062139 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.062241 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.062580 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:02.062669 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:02.062738 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:02.062856 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 12:10:02.062920 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:10:02.063282 18447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.058540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35473 (local address 127.4.48.193:51401)
0811 12:10:02.059069 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:02.059084 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:02.059202 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:02.059461 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:02.059465 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:02.059475 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 12:10:02.059751 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:02.059761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.060660 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.060665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:02.061631 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:02.061642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.062286 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.062292 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:02.062417 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.062449 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 12:10:02.062754 (+ 305us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 12:10:02.062765 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 12:10:02.062842 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 12:10:02.062969 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":305,"threads_started":1}
I20260811 12:10:02.070955 17930 raft_consensus.cc:1261] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Refusing update from remote peer 30da138ee19744faa9139964eae721d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:02.070926 18167 raft_consensus.cc:1261] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Refusing update from remote peer 30da138ee19744faa9139964eae721d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:02.071470 18434 consensus_queue.cc:1035] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c2e38d08273473780f61c19fbc1a4a3" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 43483 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:02.071787 18429 consensus_queue.cc:1035] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aaec43f412df4079a882d5dd1b9c4c6e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 44467 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:02.118291 17384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:02.118592 17384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36264:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:02.121548 17384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 887d1b9d93d242f982cb0516898e5849: Sending DeleteTablet for 3 replicas of tablet 2e26eb5597774ab988b11b9631496947
I20260811 12:10:02.122289 17625 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e26eb5597774ab988b11b9631496947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:02 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58238
I20260811 12:10:02.122637 18129 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e26eb5597774ab988b11b9631496947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:02 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50522
I20260811 12:10:02.123036 18461 tablet_replica.cc:330] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: stopping tablet replica
I20260811 12:10:02.123037 17879 tablet_service.cc:1514] Processing DeleteTablet for tablet 2e26eb5597774ab988b11b9631496947 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:02 UTC) from {username='oryx', principal='oryx/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:48170
I20260811 12:10:02.123355 18461 raft_consensus.cc:2229] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:10:02.123756 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 17513
I20260811 12:10:02.124109 18461 raft_consensus.cc:2258] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:02.126705 18461 ts_tablet_manager.cc:1905] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:02.134128 18461 ts_tablet_manager.cc:1918] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:02.134359 18461 log.cc:1196] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/2e26eb5597774ab988b11b9631496947
I20260811 12:10:02.134886 18461 ts_tablet_manager.cc:1939] T 2e26eb5597774ab988b11b9631496947 P 30da138ee19744faa9139964eae721d7: Deleting consensus metadata
I20260811 12:10:02.136113 17355 catalog_manager.cc:4841] TS 30da138ee19744faa9139964eae721d7 (127.4.48.193:35473): tablet 2e26eb5597774ab988b11b9631496947 (table test-table [id=4e4d653570b64b8687aabdcdc16ba03b]) successfully deleted
I20260811 12:10:02.137441 18464 tablet_replica.cc:330] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: stopping tablet replica
I20260811 12:10:02.137652 18464 raft_consensus.cc:2229] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:02.137876 18464 raft_consensus.cc:2258] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:02.138778 18464 ts_tablet_manager.cc:1905] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:02.142602 18463 tablet_replica.cc:330] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: stopping tablet replica
I20260811 12:10:02.142810 18463 raft_consensus.cc:2229] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:02.143036 18463 raft_consensus.cc:2258] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:02.144076 18463 ts_tablet_manager.cc:1905] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:02.145709 18464 ts_tablet_manager.cc:1918] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:02.145848 18464 log.cc:1196] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/2e26eb5597774ab988b11b9631496947
I20260811 12:10:02.146289 18464 ts_tablet_manager.cc:1939] T 2e26eb5597774ab988b11b9631496947 P aaec43f412df4079a882d5dd1b9c4c6e: Deleting consensus metadata
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.193@KRBTEST.COM
I20260811 12:10:02.147276 17352 catalog_manager.cc:4841] TS aaec43f412df4079a882d5dd1b9c4c6e (127.4.48.194:44467): tablet 2e26eb5597774ab988b11b9631496947 (table test-table [id=4e4d653570b64b8687aabdcdc16ba03b]) successfully deleted
I20260811 12:10:02.150269 18463 ts_tablet_manager.cc:1918] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 12:10:02.150398 18463 log.cc:1196] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/2e26eb5597774ab988b11b9631496947
I20260811 12:10:02.150812 18463 ts_tablet_manager.cc:1939] T 2e26eb5597774ab988b11b9631496947 P 2c2e38d08273473780f61c19fbc1a4a3: Deleting consensus metadata
I20260811 12:10:02.151925 17350 catalog_manager.cc:4841] TS 2c2e38d08273473780f61c19fbc1a4a3 (127.4.48.195:43483): tablet 2e26eb5597774ab988b11b9631496947 (table test-table [id=4e4d653570b64b8687aabdcdc16ba03b]) successfully deleted
I20260811 12:10:02.155224 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.124869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.4.48.193:35473)
0811 12:10:02.125026 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:02.125033 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:02.134814 (+ 9781us) server_negotiation.cc:407] Connection header received
0811 12:10:02.135239 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:02.135245 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:02.135351 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:02.135433 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:02.137153 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.138284 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.141490 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.142107 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.142207 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.148252 (+ 6045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:02.148270 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:02.148293 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:02.148335 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:02.152408 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.153228 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.153235 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.153238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.153373 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.153784 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.153787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.153789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.154175 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:02.154421 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:02.154759 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 12:10:02.154929 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:10:02.155982 17743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:02.294083 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 17513
I20260811 12:10:02.324623 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 17770
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.194@KRBTEST.COM
I20260811 12:10:02.338361 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.325293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48202 (local address 127.4.48.194:44467)
0811 12:10:02.325759 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:02.325764 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:02.325781 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:10:02.325849 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:02.325852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:02.325922 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:02.326017 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:02.327178 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.327927 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.329051 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.329441 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.329539 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.333514 (+ 3975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:02.333527 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:02.333543 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:02.333572 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:02.336170 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.336744 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.336750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.336752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.336877 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.337136 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.337138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.337140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.337456 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:02.337670 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:02.337870 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:10:02.338028 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":228,"threads_started":1}
I20260811 12:10:02.338989 18000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:02.459035 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 17770
I20260811 12:10:02.488226 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 18020
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[17304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.4.48.195@KRBTEST.COM
I20260811 12:10:02.502554 18471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:02.488925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.4.48.195:43483)
0811 12:10:02.489415 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:02.489420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:02.489437 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:10:02.489503 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:02.489507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:02.489577 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:02.489663 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:02.490852 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.491572 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.492707 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:02.493178 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:02.493270 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:02.497263 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:02.497277 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:02.497295 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:02.497343 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:02.500243 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.500840 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.500846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.500849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.500968 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:02.501303 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:02.501306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:02.501309 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:02.501646 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:02.501883 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:02.502072 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 12:10:02.502280 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":256,"threads_started":1}
I20260811 12:10:02.503201 18246 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:02.597047 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 18020
I20260811 12:10:02.620101 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 17322
I20260811 12:10:02.621292 17451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:02.734689 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 17322
2026-08-11T12:10:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6014 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[18478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[18478](info): set up 2 sockets
Aug 11 12:10:02 dist-test-slave-nj21 krb5kdc[18478](info): commencing operation
krb5kdc: starting...
W20260811 12:10:03.979089 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.005s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:10:04 dist-test-slave-nj21 krb5kdc[18478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:10:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:10:04.212016 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43369
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:04.535373 18498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:04.535638 18498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:04.535772 18498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:04.542740 18498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:04.542832 18498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:04.542893 18498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:04.542950 18498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:04.542997 18498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:04.552341 18498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:43369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:04.553444 18498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:04.554634 18498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:04.565948 18512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:04.565991 18506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:04.566017 18510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:04.566314 18498 server_base.cc:1034] running on GCE node
I20260811 12:10:04.567257 18498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:04.568568 18498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:04.569756 18498 hybrid_clock.cc:648] HybridClock initialized: now 1786450204569699 us; error 41 us; skew 500 ppm
I20260811 12:10:04.570062 18498 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T12:10:04Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1900 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:10:04 dist-test-slave-nj21 krb5kdc[18521](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:10:04 dist-test-slave-nj21 krb5kdc[18521](info): set up 2 sockets
Aug 11 12:10:04 dist-test-slave-nj21 krb5kdc[18521](info): commencing operation
krb5kdc: starting...
W20260811 12:10:05.867563 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.003s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:10:06 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:10:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:10:06.099130 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43469
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:06.420848 18543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:06.421113 18543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:06.421216 18543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:06.428198 18543 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:06.428298 18543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:06.428351 18543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:06.428406 18543 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:06.428448 18543 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:06.437793 18543 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:06.438956 18543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:06.440166 18543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:06.451922 18559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:06.451948 18563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:06.451892 18561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:06.452768 18543 server_base.cc:1034] running on GCE node
I20260811 12:10:06.453480 18543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:06.454771 18543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:06.455963 18543 hybrid_clock.cc:648] HybridClock initialized: now 1786450206455908 us; error 42 us; skew 500 ppm
I20260811 12:10:06.456259 18543 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:06 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:06.463279 18543 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:10:06.464726 18543 webserver.cc:460] Webserver started at http://127.4.48.254:39091/ using document root <none> and password file <none>
I20260811 12:10:06.465382 18543 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:06.465479 18543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:06.465729 18543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:06.468709 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "c6276f9deeb141cdb5f6b1820cdc76cc"
format_stamp: "Formatted at 2026-08-11 12:10:06 on dist-test-slave-nj21"
server_key: "b716ccb9760498df6ad6f1a2067d78b3"
server_key_iv: "109330849156040bd20475e36985f7d6"
server_key_version: "encryptionkey@0"
I20260811 12:10:06.469480 18543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "c6276f9deeb141cdb5f6b1820cdc76cc"
format_stamp: "Formatted at 2026-08-11 12:10:06 on dist-test-slave-nj21"
server_key: "b716ccb9760498df6ad6f1a2067d78b3"
server_key_iv: "109330849156040bd20475e36985f7d6"
server_key_version: "encryptionkey@0"
I20260811 12:10:06.474500 18543 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:10:06.477823 18575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:06.479049 18543 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:06.479197 18543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "c6276f9deeb141cdb5f6b1820cdc76cc"
format_stamp: "Formatted at 2026-08-11 12:10:06 on dist-test-slave-nj21"
server_key: "b716ccb9760498df6ad6f1a2067d78b3"
server_key_iv: "109330849156040bd20475e36985f7d6"
server_key_version: "encryptionkey@0"
I20260811 12:10:06.479372 18543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:06.550967 18543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:06.554848 18543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:06.555085 18543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:06.572099 18543 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:43469
I20260811 12:10:06.572144 18650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:43469 every 8 connection(s)
I20260811 12:10:06.573406 18543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:10:06.577097 18651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:06.580264 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 18543
I20260811 12:10:06.580447 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:10:06.581033 4291 external_mini_cluster.cc:1428] Setting key 9d3ce6935c2eb2f540fcdb882c575299
I20260811 12:10:06.589896 18651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Bootstrap starting.
I20260811 12:10:06.593291 18651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:06.594247 18651 log.cc:824] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:10:06 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:06.597209 18651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: No bootstrap required, opened a new log
I20260811 12:10:06.604223 18663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:06.582950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.4.48.254:43469)
0811 12:10:06.583804 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:06.583819 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:06.583880 (+ 61us) server_negotiation.cc:407] Connection header received
0811 12:10:06.585760 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:06.585788 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:06.586283 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:06.586695 (+ 412us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:06.588005 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:06.589913 (+ 1908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:06.591156 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:06.592215 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:06.592387 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:06.596114 (+ 3727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:06.596125 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:06.596143 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:06.596184 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:06.599969 (+ 3785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:06.600619 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:06.600627 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:06.600635 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:06.600845 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:06.601156 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:06.601161 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:06.601164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:06.601814 (+ 650us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:06.602093 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:06.602393 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 12:10:06.602831 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":341,"threads_started":1}
I20260811 12:10:06.606775 18651 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:06.607071 18651 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:06.607143 18651 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6276f9deeb141cdb5f6b1820cdc76cc, State: Initialized, Role: FOLLOWER
I20260811 12:10:06.607589 18651 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:06.607834 18651 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:10:06.607935 18651 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:10:06.608057 18651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:06.610082 18651 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:06.610519 18651 leader_election.cc:302] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6276f9deeb141cdb5f6b1820cdc76cc; no voters:
I20260811 12:10:06.610981 18651 leader_election.cc:288] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:10:06.611116 18666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:06.611750 18666 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 LEADER]: Becoming Leader. State: Replica: c6276f9deeb141cdb5f6b1820cdc76cc, State: Running, Role: LEADER
I20260811 12:10:06.612335 18666 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:06.612862 18651 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:10:06.614997 18667 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } } }
I20260811 12:10:06.615406 18667 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:06.615290 18670 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6276f9deeb141cdb5f6b1820cdc76cc. Latest consensus state: current_term: 1 leader_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } } }
I20260811 12:10:06.615592 18670 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:06.616571 18683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:10:06.621486 18683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:10:06.634155 18683 catalog_manager.cc:1347] Generated new cluster ID: 5be4dcbc1fd545bcb83241e0ccc782f5
I20260811 12:10:06.634262 18683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:10:06.648167 18683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:10:06.649888 18683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:10:06.659351 18683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Generated new TSK 0
I20260811 12:10:06.660108 18683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:10:06.718711 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:07.043953 18711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:07.044227 18711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:07.044363 18711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:07.051317 18711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:07.051424 18711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:07.051577 18711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:10:07.061065 18711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:07.062242 18711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:07.063484 18711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:07.075925 18719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:07.075896 18720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:07.075958 18722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:07.076683 18711 server_base.cc:1034] running on GCE node
I20260811 12:10:07.077245 18711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:07.078079 18711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:07.079275 18711 hybrid_clock.cc:648] HybridClock initialized: now 1786450207079231 us; error 35 us; skew 500 ppm
I20260811 12:10:07.079558 18711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:07 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450207, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:07.082688 18711 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:10:07.084101 18711 webserver.cc:460] Webserver started at http://127.4.48.193:34163/ using document root <none> and password file <none>
I20260811 12:10:07.084743 18711 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:07.084856 18711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:07.085134 18711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:07.087893 18711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "ceb8f80d26134ff09224cf9b1eadd772"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "3e9256f3f6a5e25e9f95248ccdcd1b10"
server_key_iv: "ca2ee0a654418bd15b652dec82f54b43"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.088631 18711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "ceb8f80d26134ff09224cf9b1eadd772"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "3e9256f3f6a5e25e9f95248ccdcd1b10"
server_key_iv: "ca2ee0a654418bd15b652dec82f54b43"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.093408 18711 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:10:07.096676 18734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.097848 18711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:10:07.097993 18711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ceb8f80d26134ff09224cf9b1eadd772"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "3e9256f3f6a5e25e9f95248ccdcd1b10"
server_key_iv: "ca2ee0a654418bd15b652dec82f54b43"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.098168 18711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:07.121498 18711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:07.125308 18711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:07.125561 18711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:07.126459 18711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:07.127796 18711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:07.127875 18711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.127980 18711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:07.128032 18711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.157862 18711 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:43347
I20260811 12:10:07.157912 18918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:43347 every 8 connection(s)
I20260811 12:10:07.159185 18711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:10:07.164296 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 18711
I20260811 12:10:07.164497 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:10:07.165309 4291 external_mini_cluster.cc:1428] Setting key 14b87cd9dc8fc874b5bf0ea6e7e7313a
Aug 11 12:10:07 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450207, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:07.178890 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:07.161655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:41917 (local address 127.4.48.254:43469)
0811 12:10:07.162048 (+ 393us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:07.162053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:07.163006 (+ 953us) server_negotiation.cc:407] Connection header received
0811 12:10:07.165292 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:07.165297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:07.165363 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:07.165484 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:07.167067 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.167804 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.169272 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.169677 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.169796 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:07.173340 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:07.173345 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:07.173348 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:07.173380 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:07.175656 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:07.176615 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:07.176620 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:07.176623 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:07.176759 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:07.177185 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:07.177189 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:07.177191 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:07.177507 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:07.177660 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:07.178440 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 12:10:07.178605 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":200,"threads_started":1}
I20260811 12:10:07.179955 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:07.162160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:41917)
0811 12:10:07.162850 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:07.162886 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:07.165001 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:07.165708 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:07.165715 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:07.166320 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:07.166906 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:07.166932 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.167954 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.167958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:07.169137 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:07.169150 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.169888 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.169896 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:07.170110 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:07.170716 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:07.170738 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:07.173137 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:07.175833 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:07.175841 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:07.175854 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:07.176486 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:07.176861 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:07.176865 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:07.176868 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:07.177092 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:07.177647 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:07.177655 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:07.178056 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 12:10:07.178349 (+ 293us) client_negotiation.cc:240] Negotiation successful
0811 12:10:07.178850 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":263,"threads_started":1}
I20260811 12:10:07.181388 18921 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43469
I20260811 12:10:07.181653 18921 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:07.182363 18921 heartbeater.cc:511] Master 127.4.48.254:43469 requested a full tablet report, sending...
I20260811 12:10:07.184393 18597 ts_manager.cc:194] Registered new tserver with Master: ceb8f80d26134ff09224cf9b1eadd772 (127.4.48.193:43347)
I20260811 12:10:07.185756 18597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:41917
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:10:07.218618 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:07.542351 18934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:07.542605 18934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:07.542703 18934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:07.549628 18934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:07.549722 18934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:07.549834 18934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:10:07.559103 18934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:07.560261 18934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:07.561684 18934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:07.574703 18948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:07.574797 18950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:07.575182 18940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:07.576035 18934 server_base.cc:1034] running on GCE node
I20260811 12:10:07.576575 18934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:07.577453 18934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:07.578661 18934 hybrid_clock.cc:648] HybridClock initialized: now 1786450207578601 us; error 54 us; skew 500 ppm
I20260811 12:10:07.579005 18934 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:07 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450207, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:07.582749 18934 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:10:07.584347 18934 webserver.cc:460] Webserver started at http://127.4.48.194:33161/ using document root <none> and password file <none>
I20260811 12:10:07.585170 18934 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:07.585297 18934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:07.585624 18934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:07.589361 18934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "e5826cec28db463c831ec4200df7bde6"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "a32351ce7da6b87eefc2fd0247ccc960"
server_key_iv: "275f40b138702ee4ca14c3c133b5ae86"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.590353 18934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "e5826cec28db463c831ec4200df7bde6"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "a32351ce7da6b87eefc2fd0247ccc960"
server_key_iv: "275f40b138702ee4ca14c3c133b5ae86"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.595796 18934 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:10:07.599155 18961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.600445 18934 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:10:07.600579 18934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "e5826cec28db463c831ec4200df7bde6"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "a32351ce7da6b87eefc2fd0247ccc960"
server_key_iv: "275f40b138702ee4ca14c3c133b5ae86"
server_key_version: "encryptionkey@0"
I20260811 12:10:07.600761 18934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:07.618667 18934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:07.622217 18934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:07.622460 18934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:07.623328 18934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:07.624639 18934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:07.624709 18934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.624823 18934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:07.624872 18934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:07.653945 18934 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:43115
I20260811 12:10:07.653990 19160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:43115 every 8 connection(s)
I20260811 12:10:07.655145 18934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:10:07.658514 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 18934
I20260811 12:10:07.658691 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:10:07.659257 4291 external_mini_cluster.cc:1428] Setting key 89097be4578c9254c5e8d7286de6e34a
Aug 11 12:10:07 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450207, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:07.675545 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:07.657613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:47017 (local address 127.4.48.254:43469)
0811 12:10:07.657829 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:07.657835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:07.659038 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 12:10:07.661356 (+ 2318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:07.661361 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:07.661451 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:07.661570 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:07.662977 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.664031 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.665490 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.666035 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.666164 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:07.669338 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:07.669343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:07.669347 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:07.669381 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:07.672352 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:07.673137 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:07.673141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:07.673143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:07.673281 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:07.673733 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:07.673738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:07.673740 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:07.674009 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:07.674134 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:07.674947 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 12:10:07.675188 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:10:07.676620 19167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:07.658062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:47017)
0811 12:10:07.658888 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:07.658927 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:07.661046 (+ 2119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:07.661754 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:07.661761 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:07.662261 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:07.662803 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:07.662820 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.664164 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.664168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:07.665343 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:07.665365 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:07.666247 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:07.666252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:07.666460 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:07.667062 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:07.667088 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:07.669127 (+ 2039us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:07.672479 (+ 3352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:07.672488 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:07.672503 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:07.673033 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:07.673393 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:07.673397 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:07.673400 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:07.673619 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:07.674132 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:07.674139 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:07.674535 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 12:10:07.674818 (+ 283us) client_negotiation.cc:240] Negotiation successful
0811 12:10:07.675288 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"thread_start_us":336,"threads_started":1}
I20260811 12:10:07.678412 19163 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43469
I20260811 12:10:07.678742 19163 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:07.679592 19163 heartbeater.cc:511] Master 127.4.48.254:43469 requested a full tablet report, sending...
I20260811 12:10:07.681247 18597 ts_manager.cc:194] Registered new tserver with Master: e5826cec28db463c831ec4200df7bde6 (127.4.48.194:43115)
I20260811 12:10:07.682358 18597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:47017
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:10:07.722293 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:08.041703 19171 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:08.042057 19171 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:08.042204 19171 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:08.049876 19171 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:08.049996 19171 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:08.050153 19171 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:08.060092 19171 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:08.061312 19171 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:08.062587 19171 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:08.075093 19178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:08.075244 19181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:08.075277 19185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:08.076392 19171 server_base.cc:1034] running on GCE node
I20260811 12:10:08.076990 19171 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:08.077940 19171 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:08.079151 19171 hybrid_clock.cc:648] HybridClock initialized: now 1786450208079096 us; error 41 us; skew 500 ppm
I20260811 12:10:08.079481 19171 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:08 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450208, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:08.082664 19171 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:10:08.084081 19171 webserver.cc:460] Webserver started at http://127.4.48.195:43345/ using document root <none> and password file <none>
I20260811 12:10:08.084726 19171 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:08.084842 19171 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:08.085131 19171 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:08.087916 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "778cff43004d45eebd83d9fa0d3f7715"
format_stamp: "Formatted at 2026-08-11 12:10:08 on dist-test-slave-nj21"
server_key: "fabf2a1e35928627683d6baac4b58dfc"
server_key_iv: "19c55357db4917a2651a4e196b6b2bd0"
server_key_version: "encryptionkey@0"
I20260811 12:10:08.088647 19171 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "778cff43004d45eebd83d9fa0d3f7715"
format_stamp: "Formatted at 2026-08-11 12:10:08 on dist-test-slave-nj21"
server_key: "fabf2a1e35928627683d6baac4b58dfc"
server_key_iv: "19c55357db4917a2651a4e196b6b2bd0"
server_key_version: "encryptionkey@0"
I20260811 12:10:08.093408 19171 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:10:08.096508 19194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:08.097678 19171 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:10:08.097813 19171 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "778cff43004d45eebd83d9fa0d3f7715"
format_stamp: "Formatted at 2026-08-11 12:10:08 on dist-test-slave-nj21"
server_key: "fabf2a1e35928627683d6baac4b58dfc"
server_key_iv: "19c55357db4917a2651a4e196b6b2bd0"
server_key_version: "encryptionkey@0"
I20260811 12:10:08.097970 19171 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:08.115058 19171 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:08.118903 19171 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:08.119141 19171 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:08.120074 19171 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:08.121361 19171 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:08.121433 19171 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:08.121523 19171 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:08.121564 19171 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:08.151825 19171 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44485
I20260811 12:10:08.151901 19396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44485 every 8 connection(s)
I20260811 12:10:08.153156 19171 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:08.155961 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 19171
I20260811 12:10:08.156107 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:10:08.156574 4291 external_mini_cluster.cc:1428] Setting key d09500341fb8ac0d42174180ee9fa7d6
Aug 11 12:10:08 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450208, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:08.172457 18923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:08.155716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:52055 (local address 127.4.48.254:43469)
0811 12:10:08.155931 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:08.155937 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:08.157270 (+ 1333us) server_negotiation.cc:407] Connection header received
0811 12:10:08.159520 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:08.159525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:08.159600 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:08.159704 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:08.161274 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.162031 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.163505 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.163972 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.164081 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:08.167202 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:08.167207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:08.167211 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:08.167243 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:08.169579 (+ 2336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.170205 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.170208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.170211 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.170358 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.170776 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.170781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.170784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.171130 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:08.171274 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:08.171947 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 12:10:08.172160 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:10:08.173358 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:08.156227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:52055)
0811 12:10:08.157074 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:08.157137 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:08.159268 (+ 2131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:08.159949 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:08.159956 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:08.160522 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:08.161111 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:08.161134 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.162185 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.162190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:08.163368 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:08.163380 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.164183 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.164191 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:08.164430 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:08.164990 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:08.165011 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:08.167028 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:08.169703 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:08.169709 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:08.169719 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:08.170107 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:08.170456 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:08.170460 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:08.170463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:08.170664 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:08.171260 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:08.171267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:08.171579 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 12:10:08.171865 (+ 286us) client_negotiation.cc:240] Negotiation successful
0811 12:10:08.172347 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":313,"threads_started":1}
I20260811 12:10:08.174531 19398 heartbeater.cc:347] Connected to a master server at 127.4.48.254:43469
I20260811 12:10:08.174803 19398 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:08.175480 19398 heartbeater.cc:511] Master 127.4.48.254:43469 requested a full tablet report, sending...
I20260811 12:10:08.176826 18597 ts_manager.cc:194] Registered new tserver with Master: 778cff43004d45eebd83d9fa0d3f7715 (127.4.48.195:44485)
I20260811 12:10:08.177567 18597 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:52055
I20260811 12:10:08.184326 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:10:08.188613 18921 heartbeater.cc:503] Master 127.4.48.254:43469 was elected leader, sending a full tablet report...
WARNING: no policy specified for oryx/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
Entry for principal oryx/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab.
WARNING: no policy specified for oryx/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
Entry for principal oryx/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab.
WARNING: no policy specified for oryx/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
Entry for principal oryx/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab.
I20260811 12:10:08.329779 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 18711
Aug 11 12:10:08 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:10:08.345448 19411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:08.331076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57594 (local address 127.4.48.193:43347)
0811 12:10:08.331774 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:08.331782 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:08.331810 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:10:08.331893 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:08.331900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:08.332111 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:08.332260 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:08.333586 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.334582 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.335897 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.336444 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.336538 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:08.340315 (+ 3777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:08.340326 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:08.340341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:08.340373 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:08.342889 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.343501 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.343506 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.343509 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.343627 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.344024 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.344028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.344030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.344397 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:08.344634 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:08.344883 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:10:08.345086 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":356,"threads_started":1}
I20260811 12:10:08.346726 18913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:08.427177 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 18711
I20260811 12:10:08.447113 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 18934
Aug 11 12:10:08 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:10:08.462291 19414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:08.448010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59860 (local address 127.4.48.194:43115)
0811 12:10:08.448486 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:08.448492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:08.448518 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:10:08.448586 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:08.448593 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:08.448803 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:08.448946 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:08.450216 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.451201 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.452586 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.453124 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.453237 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:08.457147 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:08.457159 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:08.457172 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:08.457205 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:08.459647 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.460283 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.460289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.460292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.460410 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.460760 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.460763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.460765 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.461084 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:08.461394 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:08.461718 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:10:08.461984 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":222,"threads_started":1}
I20260811 12:10:08.463526 19151 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:08.543519 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 18934
I20260811 12:10:08.564061 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19171
Aug 11 12:10:08 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450206, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:10:08.580077 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:08.565047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50962 (local address 127.4.48.195:44485)
0811 12:10:08.565656 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:08.565663 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:08.565692 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:10:08.565754 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:08.565760 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:08.565967 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:08.566113 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:08.567450 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.568494 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.569783 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:08.570332 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:08.570446 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:08.574800 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:08.574811 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:08.574825 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:08.574857 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:08.577472 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.578094 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.578101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.578103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.578237 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:08.578581 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:08.578585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:08.578587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:08.578925 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:08.579218 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:08.579498 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:10:08.579771 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":319,"threads_started":1}
I20260811 12:10:08.581302 19388 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:08.661695 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 19171
I20260811 12:10:08.682173 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 18543
I20260811 12:10:08.683369 18645 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:08.784400 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 18543
I20260811 12:10:08.813200 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43469
--webserver_interface=127.4.48.254
--webserver_port=39091
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:43469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:09.138159 19423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:09.138455 19423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:09.138552 19423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:09.145720 19423 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:09.145833 19423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:09.145889 19423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:09.145946 19423 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:09.145988 19423 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:09.155803 19423 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:43469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:43469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=39091
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:09.157152 19423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:09.158433 19423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:09.169893 19430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:09.169947 19435 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:09.170006 19431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:09.170975 19423 server_base.cc:1034] running on GCE node
I20260811 12:10:09.171646 19423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:09.172983 19423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:09.174224 19423 hybrid_clock.cc:648] HybridClock initialized: now 1786450209174149 us; error 57 us; skew 500 ppm
I20260811 12:10:09.174567 19423 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450209, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:09.177644 19423 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:10:09.179059 19423 webserver.cc:460] Webserver started at http://127.4.48.254:39091/ using document root <none> and password file <none>
I20260811 12:10:09.179721 19423 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:09.179872 19423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:09.184993 19423 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:10:09.187716 19451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:09.188931 19423 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:10:09.189079 19423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "c6276f9deeb141cdb5f6b1820cdc76cc"
format_stamp: "Formatted at 2026-08-11 12:10:06 on dist-test-slave-nj21"
server_key: "b716ccb9760498df6ad6f1a2067d78b3"
server_key_iv: "109330849156040bd20475e36985f7d6"
server_key_version: "encryptionkey@0"
I20260811 12:10:09.189999 19423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:09.201910 19423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:09.205552 19423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:09.205777 19423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:09.222875 19423 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:43469
I20260811 12:10:09.222891 19529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:43469 every 8 connection(s)
I20260811 12:10:09.224308 19423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:10:09.226079 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 19423
I20260811 12:10:09.227061 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:43347
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:43469
--webserver_port=34163
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:10:09.233043 19530 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:10:09.237259 19530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Bootstrap starting.
I20260811 12:10:09.256134 19530 log.cc:824] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:09.265069 19530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:10:09.265563 19530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Bootstrap complete.
I20260811 12:10:09.274343 19530 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:09.275143 19530 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6276f9deeb141cdb5f6b1820cdc76cc, State: Initialized, Role: FOLLOWER
I20260811 12:10:09.275758 19530 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:09.275993 19530 raft_consensus.cc:388] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:10:09.276114 19530 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:10:09.276270 19530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:10:09.279913 19530 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:09.280370 19530 leader_election.cc:302] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c6276f9deeb141cdb5f6b1820cdc76cc; no voters:
I20260811 12:10:09.280969 19530 leader_election.cc:288] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:10:09.281113 19537 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:10:09.282272 19537 raft_consensus.cc:686] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [term 2 LEADER]: Becoming Leader. State: Replica: c6276f9deeb141cdb5f6b1820cdc76cc, State: Running, Role: LEADER
I20260811 12:10:09.282855 19537 consensus_queue.cc:237] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } }
I20260811 12:10:09.283216 19530 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:10:09.285529 19539 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c6276f9deeb141cdb5f6b1820cdc76cc. Latest consensus state: current_term: 2 leader_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } } }
I20260811 12:10:09.285583 19538 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c6276f9deeb141cdb5f6b1820cdc76cc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 43469 } } }
I20260811 12:10:09.285893 19539 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:09.285943 19538 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:09.296810 19548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:10:09.302383 19548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:10:09.303875 19548 catalog_manager.cc:1259] Loaded cluster ID: 5be4dcbc1fd545bcb83241e0ccc782f5
I20260811 12:10:09.304028 19548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:10:09.308107 19548 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:10:09.309691 19548 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c6276f9deeb141cdb5f6b1820cdc76cc: Loaded TSK: 0
I20260811 12:10:09.311256 19548 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:10:09.567848 19533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:09.568110 19533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:09.568240 19533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:09.575153 19533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:09.575258 19533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:09.575399 19533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:10:09.584923 19533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.193.keytab
--principal=oryx/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:43347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=34163
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:09.586109 19533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:09.587414 19533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:09.601608 19571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:09.601629 19572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:09.603048 19533 server_base.cc:1034] running on GCE node
W20260811 12:10:09.603129 19574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:09.603826 19533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:09.604763 19533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:09.605985 19533 hybrid_clock.cc:648] HybridClock initialized: now 1786450209605944 us; error 35 us; skew 500 ppm
I20260811 12:10:09.606271 19533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450209, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:09.609254 19533 init.cc:377] Logged in from keytab as oryx/127.4.48.193@KRBTEST.COM (short username oryx)
I20260811 12:10:09.610714 19533 webserver.cc:460] Webserver started at http://127.4.48.193:34163/ using document root <none> and password file <none>
I20260811 12:10:09.611349 19533 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:09.611472 19533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:09.616680 19533 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:10:09.619287 19590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:09.620594 19533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:09.620743 19533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ceb8f80d26134ff09224cf9b1eadd772"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "3e9256f3f6a5e25e9f95248ccdcd1b10"
server_key_iv: "ca2ee0a654418bd15b652dec82f54b43"
server_key_version: "encryptionkey@0"
I20260811 12:10:09.621649 19533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:09.636562 19533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:09.640492 19533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:09.640734 19533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:09.641706 19533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:09.643087 19533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:09.643173 19533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:09.643277 19533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:09.643354 19533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:09.673106 19533 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:43347
I20260811 12:10:09.673187 19793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:43347 every 8 connection(s)
I20260811 12:10:09.674525 19533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:10:09.683902 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 19533
I20260811 12:10:09.684543 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:43115
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:43469
--webserver_port=33161
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:10:09.696431 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:09.678039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:43965 (local address 127.4.48.254:43469)
0811 12:10:09.684193 (+ 6154us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:09.684205 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:09.684242 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:10:09.684623 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:09.684642 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:09.685012 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:09.685351 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:09.686976 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.688427 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.689949 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.690591 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.690767 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.695605 (+ 4838us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:43965: BlockingRecv error: recv got EOF from 127.4.48.193:43965 (error 108)
Metrics: {"server-negotiator.queue_time_us":5897,"thread_start_us":5656,"threads_started":1}
I20260811 12:10:09.696430 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:09.678309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:43965)
0811 12:10:09.679233 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:09.679286 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:09.681815 (+ 2529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:09.685513 (+ 3698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:09.685524 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:09.686137 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:09.686718 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.686739 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.688590 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.688595 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.689797 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.689811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.690829 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.690838 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.691081 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.691725 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:09.691750 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:09.695256 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":364,"threads_started":1}
W20260811 12:10:09.697378 19797 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:43469 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.48.254:43469: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:09.705123 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:09.697924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:58521)
0811 12:10:09.698071 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:09.698089 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:09.698179 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:09.698448 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:09.698453 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:09.698755 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:09.698892 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.698900 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.699876 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.699880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.700732 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.700743 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.701394 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.701400 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.701555 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.702040 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:09.702055 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:09.704933 (+ 2878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 12:10:09.705403 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:09.698030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:58521 (local address 127.4.48.254:43469)
0811 12:10:09.698177 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:09.698182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:09.698200 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:10:09.698257 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:09.698262 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:09.698334 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:09.698409 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:09.699021 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.699739 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.700864 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.701255 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.701368 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.705175 (+ 3807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:58521: BlockingRecv error: recv got EOF from 127.4.48.193:58521 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:09 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:09.713351 19805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:09.705980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:43317)
0811 12:10:09.706130 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:09.706146 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:09.706219 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:09.706629 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:09.706631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:09.706897 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:09.707032 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.707039 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.708345 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.708349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.709220 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:09.709230 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.710070 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.710074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:09.710210 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.710728 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:09.710743 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:09.713154 (+ 2411us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:10:09.713647 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:09.706194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:43317 (local address 127.4.48.254:43469)
0811 12:10:09.706360 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:09.706365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:09.706384 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:10:09.706432 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:09.706437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:09.706517 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:09.706623 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:09.707157 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.708218 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.709354 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:09.709950 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:09.710081 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:09.713391 (+ 3310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:43317: BlockingRecv error: recv got EOF from 127.4.48.193:43317 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 12:10:09.713644 19797 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:10:10.014058 19806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:10.014326 19806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:10.014475 19806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:10.021720 19806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:10.021829 19806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:10.021997 19806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:10:10.031888 19806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.194.keytab
--principal=oryx/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:43115
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=33161
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:10.033075 19806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:10.034315 19806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:10.046620 19811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:10.046576 19813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:10.046552 19817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:10.047268 19806 server_base.cc:1034] running on GCE node
I20260811 12:10:10.048019 19806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:10.049225 19806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:10.050554 19806 hybrid_clock.cc:648] HybridClock initialized: now 1786450210050429 us; error 94 us; skew 500 ppm
I20260811 12:10:10.050860 19806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450210, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:10.053937 19806 init.cc:377] Logged in from keytab as oryx/127.4.48.194@KRBTEST.COM (short username oryx)
I20260811 12:10:10.055277 19806 webserver.cc:460] Webserver started at http://127.4.48.194:33161/ using document root <none> and password file <none>
I20260811 12:10:10.056007 19806 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:10.056121 19806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:10.061228 19806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:10:10.063763 19829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.065039 19806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:10:10.065176 19806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "e5826cec28db463c831ec4200df7bde6"
format_stamp: "Formatted at 2026-08-11 12:10:07 on dist-test-slave-nj21"
server_key: "a32351ce7da6b87eefc2fd0247ccc960"
server_key_iv: "275f40b138702ee4ca14c3c133b5ae86"
server_key_version: "encryptionkey@0"
I20260811 12:10:10.066052 19806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:10.083200 19806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:10.086786 19806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:10.086992 19806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:10.087869 19806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:10.089095 19806 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:10.089195 19806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.089300 19806 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:10.089368 19806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.119656 19806 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:43115
I20260811 12:10:10.119738 20022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:43115 every 8 connection(s)
I20260811 12:10:10.121065 19806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:10:10.130750 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 19806
I20260811 12:10:10.131434 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:44485
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:43469
--webserver_port=43345
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:45987
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:10:10.136826 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.123463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:37119 (local address 127.4.48.254:43469)
0811 12:10:10.123625 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.123629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.124790 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 12:10:10.126626 (+ 1836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.126632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.126711 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.126800 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.128286 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.128998 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.130558 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.131026 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.131127 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.136627 (+ 5500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:37119: BlockingRecv error: recv got EOF from 127.4.48.194:37119 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:10:10.137655 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.123891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:37119)
0811 12:10:10.124632 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.124674 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.126365 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.127038 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.127046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.127538 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.128123 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.128140 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.129152 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.129156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.130384 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.130400 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.131264 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.131272 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.131570 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.132240 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.132274 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.136436 (+ 4162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":269,"threads_started":1}
W20260811 12:10:10.138350 20023 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:43469 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.48.254:43469: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:10.146384 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.138871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:55713)
0811 12:10:10.139042 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.139059 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.139153 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.139428 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.139432 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.139838 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.140002 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.140012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.140929 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.140933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.142099 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.142115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.142854 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.142858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.142976 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.143490 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.143508 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.146186 (+ 2678us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:10:10.146661 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.139013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:55713 (local address 127.4.48.254:43469)
0811 12:10:10.139146 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.139150 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.139166 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:10:10.139255 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.139258 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.139314 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.139387 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.140124 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.140799 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.142216 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.142719 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.142839 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.146437 (+ 3598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:55713: BlockingRecv error: recv got EOF from 127.4.48.194:55713 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:10.154315 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.147123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:56267)
0811 12:10:10.147264 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.147282 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.147379 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.147750 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.147755 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.148032 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.148166 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.148173 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.149134 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.149139 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.150026 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.150038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.150702 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.150707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.150890 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.151397 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.151415 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.154047 (+ 2632us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 12:10:10.154469 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.147302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:56267 (local address 127.4.48.254:43469)
0811 12:10:10.147480 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.147486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.147501 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:10:10.147544 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.147547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.147599 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.147697 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.148297 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.149001 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.150157 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.150566 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.150678 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.154269 (+ 3591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:56267: BlockingRecv error: recv got EOF from 127.4.48.194:56267 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 12:10:10.154565 20023 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:10:10.457482 20029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:10.457732 20029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:10.457863 20029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:10.464920 20029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:10.465026 20029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:10.465180 20029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:10.474519 20029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45987
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/krb5kdc/oryx_127.4.48.195.keytab
--principal=oryx/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:44485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=43345
--tserver_master_addrs=127.4.48.254:43469
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:10.475744 20029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:10.476989 20029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:10.489470 20045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:10.489509 20041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:10.489753 20039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:10.490360 20029 server_base.cc:1034] running on GCE node
I20260811 12:10:10.490953 20029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:10.491844 20029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:10.493052 20029 hybrid_clock.cc:648] HybridClock initialized: now 1786450210492998 us; error 38 us; skew 500 ppm
I20260811 12:10:10.493340 20029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450210, etypes {rep=17 tkt=17 ses=17}, oryx/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:10.496284 20029 init.cc:377] Logged in from keytab as oryx/127.4.48.195@KRBTEST.COM (short username oryx)
I20260811 12:10:10.497605 20029 webserver.cc:460] Webserver started at http://127.4.48.195:43345/ using document root <none> and password file <none>
I20260811 12:10:10.498242 20029 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:10.498369 20029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:10.503415 20029 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:10:10.505955 20055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.507160 20029 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:10:10.507313 20029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "778cff43004d45eebd83d9fa0d3f7715"
format_stamp: "Formatted at 2026-08-11 12:10:08 on dist-test-slave-nj21"
server_key: "fabf2a1e35928627683d6baac4b58dfc"
server_key_iv: "19c55357db4917a2651a4e196b6b2bd0"
server_key_version: "encryptionkey@0"
I20260811 12:10:10.508270 20029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:10.523981 20029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:10.527906 20029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:10.528185 20029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:10.529081 20029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:10.530345 20029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:10.530440 20029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.530543 20029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:10.530617 20029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:10.560078 20029 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44485
I20260811 12:10:10.560143 20252 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44485 every 8 connection(s)
I20260811 12:10:10.561478 20029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:10.568986 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 20029
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
W20260811 12:10:10.577386 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.563921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35789 (local address 127.4.48.254:43469)
0811 12:10:10.564101 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.564105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.565223 (+ 1118us) server_negotiation.cc:407] Connection header received
0811 12:10:10.567103 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.567108 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.567193 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.567278 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.568841 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.569596 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.571711 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.572359 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.572492 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.577123 (+ 4631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:35789: BlockingRecv error: recv got EOF from 127.4.48.195:35789 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:10:10.577886 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.564375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:35789)
0811 12:10:10.565069 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.565113 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.566826 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.567519 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.567528 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.568068 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.568619 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.568637 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.569775 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.569781 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.571524 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.571536 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.572546 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.572551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.572784 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.573481 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.573511 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.576866 (+ 3355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":278,"threads_started":1}
W20260811 12:10:10.578510 20253 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:43469 (0 consecutive failures): Not authorized: Failed to ping master at 127.4.48.254:43469: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
I20260811 12:10:10.584988 20259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.569977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.4.48.254:43469)
0811 12:10:10.570702 (+ 725us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.570707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.570725 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:10:10.570774 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.570779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.570850 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.570968 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.572739 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.573746 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.574969 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.575372 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.575477 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.578276 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:10.578284 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:10.578298 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:10.578335 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:10.581787 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:10.582415 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:10.582423 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:10.582433 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:10.582625 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:10.582992 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:10.582996 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:10.582999 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:10.583581 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:10.583925 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:10.584270 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 12:10:10.584615 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":493,"threads_started":1}
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:10.587016 20256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.579031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:35265)
0811 12:10:10.579174 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.579191 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.579264 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.579532 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.579534 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.579850 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.579977 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.579985 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.581006 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.581011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.582199 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.582211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.583037 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.583043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.583212 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.583727 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.583749 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.586774 (+ 3025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 12:10:10.587478 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.579108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35265 (local address 127.4.48.254:43469)
0811 12:10:10.579261 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.579266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.579283 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:10:10.579338 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.579343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.579420 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.579525 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.580106 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.580871 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.582339 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.582888 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.583020 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.586954 (+ 3934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:35265: BlockingRecv error: recv got EOF from 127.4.48.195:35265 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:10.594600 20262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.587615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:48037)
0811 12:10:10.588227 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.588244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.588337 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.588614 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.588619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.589011 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.589199 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.589207 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.590051 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.590054 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.590935 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.590946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.591514 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.591518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.591620 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.592068 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.592084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.594399 (+ 2315us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":304,"threads_started":1}
W20260811 12:10:10.594726 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.587697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:48037 (local address 127.4.48.254:43469)
0811 12:10:10.588119 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.588124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.588313 (+ 189us) server_negotiation.cc:407] Connection header received
0811 12:10:10.588409 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.588413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.588477 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.588563 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.589302 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.589952 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.591032 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.591392 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.591474 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.594555 (+ 3081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:48037: BlockingRecv error: recv got EOF from 127.4.48.195:48037 (error 108)
Metrics: {"server-negotiator.queue_time_us":165}
W20260811 12:10:10.594880 20253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:10 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:10.722563 20263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:10.714207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:59809)
0811 12:10:10.714720 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:10.714736 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:10.714879 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:10.715234 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:10.715240 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:10.715901 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:10.716139 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.716151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.717216 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.717220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.718168 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:10.718180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.719020 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.719025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:10.719148 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.719729 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:10.719748 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:10.722339 (+ 2591us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":254,"threads_started":1}
W20260811 12:10:10.722716 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:10.714290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:59809 (local address 127.4.48.254:43469)
0811 12:10:10.714525 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:10.714532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:10.714824 (+ 292us) server_negotiation.cc:407] Connection header received
0811 12:10:10.715024 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:10.715028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:10.715093 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:10.715167 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:10.716270 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.717076 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.718284 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:10.718858 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:10.718969 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:10.722520 (+ 3551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:59809: BlockingRecv error: recv got EOF from 127.4.48.193:59809 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:11.162703 20264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:11.155125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:42901)
0811 12:10:11.155625 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:11.155645 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:11.155774 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:11.156020 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:11.156023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:11.156494 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:11.156688 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.156697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.157590 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.157593 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.158532 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.158544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.159152 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.159156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.159273 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.159865 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:11.159884 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:11.162473 (+ 2589us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":263,"threads_started":1}
W20260811 12:10:11.162878 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:11.155150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:42901 (local address 127.4.48.254:43469)
0811 12:10:11.155328 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:11.155332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:11.155725 (+ 393us) server_negotiation.cc:407] Connection header received
0811 12:10:11.155854 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:11.155857 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:11.155916 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:11.155992 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:11.156795 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.157485 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.158633 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.159024 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.159108 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.162658 (+ 3550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:42901: BlockingRecv error: recv got EOF from 127.4.48.194:42901 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:11.607946 20265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:11.595395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:51807)
0811 12:10:11.595879 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:11.595896 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:11.596005 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:11.596258 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:11.596261 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:11.596686 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:11.596882 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.596891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.597836 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.597840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.598749 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.598761 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.599355 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.599359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.599477 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.605097 (+ 5620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:11.605115 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:11.607720 (+ 2605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":231,"threads_started":1}
W20260811 12:10:11.608111 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:11.595435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:51807 (local address 127.4.48.254:43469)
0811 12:10:11.595641 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:11.595648 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:11.595968 (+ 320us) server_negotiation.cc:407] Connection header received
0811 12:10:11.596094 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:11.596097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:11.596158 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:11.596233 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:11.596992 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.597708 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.598858 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.599230 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.599313 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.607907 (+ 8594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:51807: BlockingRecv error: recv got EOF from 127.4.48.195:51807 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:11 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:11.731436 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:11.723396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:37389)
0811 12:10:11.723916 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:11.723933 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:11.724045 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:11.724364 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:11.724367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:11.724781 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:11.724947 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.724958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.726002 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.726006 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.726911 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:11.726922 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.727548 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.727551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:11.727657 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.728189 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:11.728206 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:11.731188 (+ 2982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":281,"threads_started":1}
W20260811 12:10:11.731598 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:11.723454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37389 (local address 127.4.48.254:43469)
0811 12:10:11.723664 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:11.723693 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:11.724029 (+ 336us) server_negotiation.cc:407] Connection header received
0811 12:10:11.724147 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:11.724151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:11.724229 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:11.724317 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:11.725101 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.725873 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.727034 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:11.727429 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:11.727524 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:11.731379 (+ 3855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:37389: BlockingRecv error: recv got EOF from 127.4.48.193:37389 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:12.172493 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:12.163541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:58315)
0811 12:10:12.164100 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:12.164123 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:12.164277 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:12.164662 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:12.164667 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:12.165132 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:12.165318 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.165328 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.166420 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.166425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.167390 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.167403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.168252 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.168259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.168449 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.169075 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:12.169104 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:12.172235 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":315,"threads_started":1}
W20260811 12:10:12.172652 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:12.163721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:58315 (local address 127.4.48.254:43469)
0811 12:10:12.163958 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:12.163965 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:12.164208 (+ 243us) server_negotiation.cc:407] Connection header received
0811 12:10:12.164415 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:12.164421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:12.164518 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:12.164612 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:12.165468 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.166270 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.167537 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.168080 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.168218 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.172458 (+ 4240us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:58315: BlockingRecv error: recv got EOF from 127.4.48.194:58315 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:12.616899 20271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:12.608744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:39171)
0811 12:10:12.609236 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:12.609254 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:12.609377 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:12.609651 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:12.609655 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:12.610088 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:12.610267 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.610278 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.611247 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.611251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.612280 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.612295 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.613001 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.613006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.613124 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.613737 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:12.613757 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:12.616651 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":236,"threads_started":1}
W20260811 12:10:12.617008 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:12.608973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:39171 (local address 127.4.48.254:43469)
0811 12:10:12.609172 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:12.609178 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:12.609321 (+ 143us) server_negotiation.cc:407] Connection header received
0811 12:10:12.609475 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:12.609478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:12.609540 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:12.609615 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:12.610382 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.611138 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.612405 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.612868 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.612960 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.616789 (+ 3829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:39171: BlockingRecv error: recv got EOF from 127.4.48.195:39171 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:12 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:12.740521 20272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:12.732342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:37379)
0811 12:10:12.732875 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:12.732892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:12.733011 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:12.733277 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:12.733280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:12.733619 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:12.733814 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.733823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.735079 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.735084 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.736227 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:12.736244 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.736866 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.736870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:12.736994 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.737593 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:12.737612 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:12.740267 (+ 2655us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":275,"threads_started":1}
W20260811 12:10:12.740674 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:12.732460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37379 (local address 127.4.48.254:43469)
0811 12:10:12.732636 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:12.732641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:12.732955 (+ 314us) server_negotiation.cc:407] Connection header received
0811 12:10:12.733095 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:12.733098 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:12.733159 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:12.733242 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:12.733973 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.734908 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.736362 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:12.736750 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:12.736846 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:12.740458 (+ 3612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:37379: BlockingRecv error: recv got EOF from 127.4.48.193:37379 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:13.181180 20273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:13.173315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:34245)
0811 12:10:13.173739 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:13.173759 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:13.173886 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:13.174162 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:13.174165 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:13.174514 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:13.174704 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.174718 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.176091 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.176095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.176988 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.177001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.177586 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.177590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.177708 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.178216 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:13.178232 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:13.180935 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":222,"threads_started":1}
W20260811 12:10:13.181313 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:13.173402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:34245 (local address 127.4.48.254:43469)
0811 12:10:13.173608 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:13.173615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:13.173833 (+ 218us) server_negotiation.cc:407] Connection header received
0811 12:10:13.173975 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:13.173979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:13.174042 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:13.174118 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:13.174864 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.175966 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.177095 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.177487 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.177585 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.181114 (+ 3529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:34245: BlockingRecv error: recv got EOF from 127.4.48.194:34245 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:13.625672 20274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:13.617855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:52243)
0811 12:10:13.618307 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:13.618322 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:13.618437 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:13.618709 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:13.618712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:13.619061 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:13.619245 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.619253 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.620186 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.620189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.621089 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.621101 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.621682 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.621686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.621800 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.622335 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:13.622355 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:13.625335 (+ 2980us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":238,"threads_started":1}
W20260811 12:10:13.625782 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:13.617912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:52243 (local address 127.4.48.254:43469)
0811 12:10:13.618078 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:13.618083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:13.618387 (+ 304us) server_negotiation.cc:407] Connection header received
0811 12:10:13.618539 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:13.618542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:13.618603 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:13.618680 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:13.619365 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.620082 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.621191 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.621583 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.621669 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.625570 (+ 3901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:52243: BlockingRecv error: recv got EOF from 127.4.48.195:52243 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:13 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:13.749301 20275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:13.741627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:51749)
0811 12:10:13.742145 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:13.742162 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:13.742285 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:13.742552 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:13.742555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:13.742950 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:13.743109 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.743118 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.744081 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.744085 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.745049 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:13.745061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.745657 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.745661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:13.745802 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.746371 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:13.746390 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:13.749066 (+ 2676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":281,"threads_started":1}
W20260811 12:10:13.749521 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:13.741648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:51749 (local address 127.4.48.254:43469)
0811 12:10:13.741851 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:13.741855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:13.742231 (+ 376us) server_negotiation.cc:407] Connection header received
0811 12:10:13.742373 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:13.742376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:13.742435 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:13.742509 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:13.743218 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.743980 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.745164 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:13.745560 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:13.745642 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:13.749257 (+ 3615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:51749: BlockingRecv error: recv got EOF from 127.4.48.193:51749 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:14.190181 20276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:14.182101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:42451)
0811 12:10:14.182583 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:14.182604 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:14.182745 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:14.183063 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:14.183067 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:14.183513 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:14.183724 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.183737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.184806 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.184812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.185760 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.185781 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.186469 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.186475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.186652 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.187225 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:14.187243 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:14.189877 (+ 2634us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":249,"threads_started":1}
W20260811 12:10:14.190321 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:14.182144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:42451 (local address 127.4.48.254:43469)
0811 12:10:14.182324 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:14.182330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:14.182685 (+ 355us) server_negotiation.cc:407] Connection header received
0811 12:10:14.182867 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:14.182873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:14.182938 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:14.183027 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:14.183867 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.184675 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.185902 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.186315 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.186443 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.190086 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:42451: BlockingRecv error: recv got EOF from 127.4.48.194:42451 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:14.634428 20277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:14.626681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:37867)
0811 12:10:14.627171 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:14.627187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:14.627312 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:14.627577 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:14.627579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:14.627992 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:14.628148 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.628157 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.629089 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.629092 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.630058 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.630069 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.630651 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.630655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.630779 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.631351 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:14.631378 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:14.634192 (+ 2814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":241,"threads_started":1}
W20260811 12:10:14.634604 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:14.626738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:37867 (local address 127.4.48.254:43469)
0811 12:10:14.626896 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:14.626900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:14.627260 (+ 360us) server_negotiation.cc:407] Connection header received
0811 12:10:14.627396 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:14.627400 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:14.627461 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:14.627537 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:14.628253 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.628991 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.630161 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.630551 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.630638 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.634377 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:37867: BlockingRecv error: recv got EOF from 127.4.48.195:37867 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:14 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:14.757627 20278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:14.750219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:51731)
0811 12:10:14.750696 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:14.750713 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:14.750831 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:14.751086 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:14.751090 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:14.751505 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:14.751716 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.751726 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.752646 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.752650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.753549 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:14.753560 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.754154 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.754158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:14.754270 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.754787 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:14.754804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:14.757385 (+ 2581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":231,"threads_started":1}
W20260811 12:10:14.757735 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:14.750265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:51731 (local address 127.4.48.254:43469)
0811 12:10:14.750476 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:14.750483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:14.750783 (+ 300us) server_negotiation.cc:407] Connection header received
0811 12:10:14.750918 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:14.750921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:14.750982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:14.751057 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:14.751828 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.752545 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.753648 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:14.754055 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:14.754139 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:14.757555 (+ 3416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:51731: BlockingRecv error: recv got EOF from 127.4.48.193:51731 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:15.199363 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:15.191145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:41941)
0811 12:10:15.191648 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:15.191693 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:15.191803 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:15.192075 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:15.192078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:15.192472 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:15.192637 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.192645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.193681 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.193685 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.194764 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.194778 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.195444 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.195448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.195558 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.196125 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:15.196142 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:15.199026 (+ 2884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":232,"threads_started":1}
W20260811 12:10:15.199488 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:15.191290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:41941 (local address 127.4.48.254:43469)
0811 12:10:15.191454 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:15.191459 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:15.191755 (+ 296us) server_negotiation.cc:407] Connection header received
0811 12:10:15.191912 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:15.191915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:15.191978 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:15.192047 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:15.192747 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.193555 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.194878 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.195330 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.195429 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.199220 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:41941: BlockingRecv error: recv got EOF from 127.4.48.194:41941 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:15.642992 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:15.635295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:45991)
0811 12:10:15.635872 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:15.635892 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:15.636034 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:15.636367 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:15.636371 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:15.636842 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:15.637009 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.637019 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.637981 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.637985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.638893 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.638904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.639502 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.639506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.639624 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.640158 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:15.640175 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:15.642732 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":239,"threads_started":1}
W20260811 12:10:15.643121 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:15.635355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:45991 (local address 127.4.48.254:43469)
0811 12:10:15.635570 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:15.635576 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:15.635975 (+ 399us) server_negotiation.cc:407] Connection header received
0811 12:10:15.636151 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:15.636155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:15.636218 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:15.636301 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:15.637136 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.637847 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.638997 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.639401 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.639488 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.642914 (+ 3426us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:45991: BlockingRecv error: recv got EOF from 127.4.48.195:45991 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:15 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:15.766469 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:15.758495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:37183)
0811 12:10:15.759038 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:15.759054 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:15.759163 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:15.759449 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:15.759451 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:15.759872 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:15.760038 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.760046 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.760977 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.760980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.761851 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:15.761861 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.762749 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.762755 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:15.762893 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.763516 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:15.763539 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:15.766243 (+ 2704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":292,"threads_started":1}
W20260811 12:10:15.766649 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:15.758561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37183 (local address 127.4.48.254:43469)
0811 12:10:15.758712 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:15.758716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:15.759118 (+ 402us) server_negotiation.cc:407] Connection header received
0811 12:10:15.759263 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:15.759267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:15.759340 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:15.759420 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:15.760144 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.760872 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.761990 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:15.762586 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:15.762711 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:15.766423 (+ 3712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:37183: BlockingRecv error: recv got EOF from 127.4.48.193:37183 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:16.208230 20284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:16.200394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:59109)
0811 12:10:16.200913 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:16.200935 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:16.201049 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:16.201385 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:16.201390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:16.201788 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:16.201958 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.201968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.202948 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.202952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.203876 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.203889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.204580 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.204586 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.204756 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.205289 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:16.205306 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:16.207930 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
W20260811 12:10:16.208433 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:16.200437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:59109 (local address 127.4.48.254:43469)
0811 12:10:16.200635 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:16.200641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:16.201011 (+ 370us) server_negotiation.cc:407] Connection header received
0811 12:10:16.201176 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:16.201182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:16.201250 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:16.201340 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:16.202097 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.202811 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.204011 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.204423 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.204541 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.208174 (+ 3633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:59109: BlockingRecv error: recv got EOF from 127.4.48.194:59109 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:16.655900 20285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:16.643879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:50379)
0811 12:10:16.644370 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:16.644386 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:16.644517 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:16.644839 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:16.644844 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:16.645246 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:16.645399 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.645409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.646425 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.646429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.647322 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.647334 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.648113 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.648119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.648272 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.652595 (+ 4323us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:16.652619 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:16.655543 (+ 2924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":260,"threads_started":1}
W20260811 12:10:16.656038 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:16.643941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:50379 (local address 127.4.48.254:43469)
0811 12:10:16.644132 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:16.644138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:16.644469 (+ 331us) server_negotiation.cc:407] Connection header received
0811 12:10:16.644627 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:16.644632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:16.644699 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:16.644780 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:16.645544 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.646283 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.647465 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.647952 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.648063 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.655802 (+ 7739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:50379: BlockingRecv error: recv got EOF from 127.4.48.195:50379 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:16 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:16.775283 20286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:16.767249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:47267)
0811 12:10:16.767859 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:16.767876 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:16.767988 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:16.768269 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:16.768272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:16.768664 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:16.768835 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.768847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.769861 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.769864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.770920 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:16.770934 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.771776 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.771781 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:16.771902 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.772469 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:16.772487 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:16.775034 (+ 2547us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":264,"threads_started":1}
W20260811 12:10:16.775411 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:16.767370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:47267 (local address 127.4.48.254:43469)
0811 12:10:16.767544 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:16.767548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:16.767935 (+ 387us) server_negotiation.cc:407] Connection header received
0811 12:10:16.768069 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:16.768073 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:16.768151 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:16.768238 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:16.768943 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.769736 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.771044 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:16.771595 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:16.771719 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:16.775228 (+ 3509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:47267: BlockingRecv error: recv got EOF from 127.4.48.193:47267 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:17.217515 20288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:17.209161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:49673)
0811 12:10:17.209751 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:17.209771 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:17.209908 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:17.210258 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:17.210263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:17.210816 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:17.211031 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.211045 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.212189 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.212193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.213149 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.213161 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.213819 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.213822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.213929 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.214484 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:17.214501 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:17.217262 (+ 2761us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":282,"threads_started":1}
W20260811 12:10:17.217692 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:17.209229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:49673 (local address 127.4.48.254:43469)
0811 12:10:17.209436 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:17.209451 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:17.209858 (+ 407us) server_negotiation.cc:407] Connection header received
0811 12:10:17.210024 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:17.210027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:17.210102 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:17.210209 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:17.211176 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.212070 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.213264 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.213686 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.213778 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.217461 (+ 3683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:49673: BlockingRecv error: recv got EOF from 127.4.48.194:49673 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:17.665117 20292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:17.656855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:34663)
0811 12:10:17.657403 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:17.657420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:17.657540 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:17.657867 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:17.657871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:17.658239 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:17.658426 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.658435 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.659395 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.659400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.660533 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.660548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.661455 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.661462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.661594 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.662157 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:17.662175 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:17.664867 (+ 2692us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":224,"threads_started":1}
W20260811 12:10:17.665222 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:17.656921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:34663 (local address 127.4.48.254:43469)
0811 12:10:17.657114 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:17.657122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:17.657497 (+ 375us) server_negotiation.cc:407] Connection header received
0811 12:10:17.657667 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:17.657671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:17.657738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:17.657828 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:17.658533 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.659249 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.660655 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.661278 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.661410 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.665042 (+ 3632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:34663: BlockingRecv error: recv got EOF from 127.4.48.195:34663 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:17 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:17.784315 20293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:17.776222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:50545)
0811 12:10:17.776749 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:17.776768 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:17.776908 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:17.777250 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:17.777256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:17.777689 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:17.777846 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.777855 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.778837 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.778841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.779822 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:17.779835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.780580 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.780587 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:17.780737 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.781256 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:17.781273 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:17.784004 (+ 2731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":387,"threads_started":1}
W20260811 12:10:17.784551 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:17.776420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:50545 (local address 127.4.48.254:43469)
0811 12:10:17.776612 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:17.776618 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:17.776864 (+ 246us) server_negotiation.cc:407] Connection header received
0811 12:10:17.777047 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:17.777052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:17.777119 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:17.777198 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:17.777989 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.778691 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.779955 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:17.780406 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:17.780515 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:17.784187 (+ 3672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:50545: BlockingRecv error: recv got EOF from 127.4.48.193:50545 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:18.226311 20295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:18.218327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:42715)
0811 12:10:18.218822 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:18.218844 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:18.218958 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:18.219247 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:18.219250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:18.219652 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:18.219835 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.219847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.220931 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.220935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.221819 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.221830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.222474 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.222477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.222588 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.223158 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:18.223184 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:18.226077 (+ 2893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":244,"threads_started":1}
W20260811 12:10:18.226544 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:18.218398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:42715 (local address 127.4.48.254:43469)
0811 12:10:18.218558 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:18.218562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:18.218920 (+ 358us) server_negotiation.cc:407] Connection header received
0811 12:10:18.219061 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:18.219067 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:18.219139 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:18.219220 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:18.219953 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.220802 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.221919 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.222345 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.222430 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.226296 (+ 3866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:42715: BlockingRecv error: recv got EOF from 127.4.48.194:42715 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:18.674271 20296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:18.666094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:41533)
0811 12:10:18.666617 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:18.666634 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:18.666791 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:18.667081 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:18.667085 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:18.667487 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:18.667639 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.667649 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.668702 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.668706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.669757 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.669775 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.670656 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.670663 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.670820 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.671410 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:18.671428 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:18.674050 (+ 2622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":273,"threads_started":1}
W20260811 12:10:18.674540 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:18.666210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:41533 (local address 127.4.48.254:43469)
0811 12:10:18.666439 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:18.666445 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:18.666729 (+ 284us) server_negotiation.cc:407] Connection header received
0811 12:10:18.666886 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:18.666890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:18.666953 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:18.667031 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:18.667811 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.668513 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.669931 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.670480 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.670592 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.674294 (+ 3702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:41533: BlockingRecv error: recv got EOF from 127.4.48.195:41533 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:18 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:18.793365 20297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:18.785151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:47447)
0811 12:10:18.785661 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:18.785677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:18.785798 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:18.786099 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:18.786104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:18.786607 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:18.786898 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.786915 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.788200 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.788205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.789129 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:18.789140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.789799 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.789804 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:18.789921 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.790475 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:18.790493 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:18.793113 (+ 2620us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":266,"threads_started":1}
W20260811 12:10:18.793563 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:18.785207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:47447 (local address 127.4.48.254:43469)
0811 12:10:18.785371 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:18.785375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:18.785741 (+ 366us) server_negotiation.cc:407] Connection header received
0811 12:10:18.785885 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:18.785889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:18.785967 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:18.786047 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:18.787047 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.788078 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.789247 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:18.789666 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:18.789757 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:18.793309 (+ 3552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:47447: BlockingRecv error: recv got EOF from 127.4.48.193:47447 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:19.235821 20298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:19.227262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:39639)
0811 12:10:19.227812 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:19.227833 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:19.227981 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:19.228360 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:19.228366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:19.229037 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:19.229268 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.229282 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.230461 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.230466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.231447 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.231460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.232178 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.232184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.232359 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.232873 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:19.232890 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:19.235462 (+ 2572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":289,"threads_started":1}
W20260811 12:10:19.235924 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:19.227357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:39639 (local address 127.4.48.254:43469)
0811 12:10:19.227543 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:19.227549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:19.227913 (+ 364us) server_negotiation.cc:407] Connection header received
0811 12:10:19.228099 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:19.228105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:19.228181 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:19.228280 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:19.229410 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.230320 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.231580 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.232028 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.232138 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.235697 (+ 3559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:39639: BlockingRecv error: recv got EOF from 127.4.48.194:39639 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:19.684880 20301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:19.675238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:39883)
0811 12:10:19.675920 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:19.675941 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:19.676073 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:19.676422 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:19.676426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:19.676953 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:19.677182 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.677196 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.678531 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.678536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.679795 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.679813 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.681011 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.681017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.681198 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.681814 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:19.681847 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:19.684623 (+ 2776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":410,"threads_started":1}
W20260811 12:10:19.685007 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:19.675370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:39883 (local address 127.4.48.254:43469)
0811 12:10:19.675973 (+ 603us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:19.675980 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:19.676000 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:10:19.676201 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:19.676204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:19.676283 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:19.676379 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:19.677323 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.678402 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.679999 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.680617 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.680759 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.684810 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:39883: BlockingRecv error: recv got EOF from 127.4.48.195:39883 (error 108)
Metrics: {"server-negotiator.queue_time_us":472}
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:19 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:19.802155 20303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:19.794219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:53929)
0811 12:10:19.794736 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:19.794752 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:19.794881 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:19.795212 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:19.795217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:19.795719 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:19.795886 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.795896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.796939 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.796945 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.797854 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:19.797867 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.798582 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.798588 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:19.798752 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.799312 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:19.799330 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:19.801838 (+ 2508us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":281,"threads_started":1}
W20260811 12:10:19.802299 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:19.794330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:53929 (local address 127.4.48.254:43469)
0811 12:10:19.794533 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:19.794539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:19.794846 (+ 307us) server_negotiation.cc:407] Connection header received
0811 12:10:19.794995 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:19.795000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:19.795084 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:19.795165 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:19.796030 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.796804 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.797991 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:19.798437 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:19.798546 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:19.802069 (+ 3523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:53929: BlockingRecv error: recv got EOF from 127.4.48.193:53929 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:20.244478 20306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:20.236776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:48611)
0811 12:10:20.237290 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:20.237312 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:20.237442 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:20.237787 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:20.237791 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:20.238230 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:20.238463 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.238478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.239415 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.239419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.240418 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.240431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.241052 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.241057 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.241174 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.241701 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:20.241719 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:20.244236 (+ 2517us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":277,"threads_started":1}
W20260811 12:10:20.244622 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:20.236855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:48611 (local address 127.4.48.254:43469)
0811 12:10:20.237063 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:20.237069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:20.237401 (+ 332us) server_negotiation.cc:407] Connection header received
0811 12:10:20.237555 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:20.237561 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:20.237641 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:20.237737 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:20.238616 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.239316 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.240529 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.240938 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.241027 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.244417 (+ 3390us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:48611: BlockingRecv error: recv got EOF from 127.4.48.194:48611 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:20.693861 20308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:20.685752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:37881)
0811 12:10:20.686239 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:20.686255 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:20.686371 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:20.686635 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:20.686640 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:20.687034 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:20.687208 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.687217 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.688282 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.688287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.689474 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.689489 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.690161 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.690166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.690286 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.690806 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:20.690824 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:20.693573 (+ 2749us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":274,"threads_started":1}
W20260811 12:10:20.694010 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:20.685889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:37881 (local address 127.4.48.254:43469)
0811 12:10:20.686040 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:20.686044 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:20.686316 (+ 272us) server_negotiation.cc:407] Connection header received
0811 12:10:20.686457 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:20.686460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:20.686519 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:20.686596 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:20.687332 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.688150 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.689628 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.690037 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.690127 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.693766 (+ 3639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:37881: BlockingRecv error: recv got EOF from 127.4.48.195:37881 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:20 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:20.811028 20309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:20.803014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:35127)
0811 12:10:20.803530 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:20.803551 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:20.803697 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:20.804057 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:20.804062 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:20.804620 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:20.804807 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.804817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.805777 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.805783 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.806669 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:20.806680 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.807377 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.807383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:20.807537 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.808072 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:20.808088 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:20.810710 (+ 2622us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":235,"threads_started":1}
W20260811 12:10:20.811304 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:20.803091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:35127 (local address 127.4.48.254:43469)
0811 12:10:20.803288 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:20.803294 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:20.803631 (+ 337us) server_negotiation.cc:407] Connection header received
0811 12:10:20.803810 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:20.803813 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:20.803910 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:20.804001 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:20.804943 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.805648 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.806798 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:20.807227 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:20.807334 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:20.810929 (+ 3595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:35127: BlockingRecv error: recv got EOF from 127.4.48.193:35127 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:21.252728 20310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:21.245373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:53903)
0811 12:10:21.245824 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:21.245845 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:21.245965 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:21.246214 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:21.246217 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:21.246621 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:21.246795 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.246803 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.247737 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.247741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.248625 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.248636 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.249236 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.249240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.249348 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.249877 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:21.249894 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:21.252492 (+ 2598us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":220,"threads_started":1}
W20260811 12:10:21.252907 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:21.245425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:53903 (local address 127.4.48.254:43469)
0811 12:10:21.245616 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:21.245623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:21.245919 (+ 296us) server_negotiation.cc:407] Connection header received
0811 12:10:21.246046 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:21.246050 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:21.246111 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:21.246183 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:21.246912 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.247607 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.248727 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.249129 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.249213 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.252698 (+ 3485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:53903: BlockingRecv error: recv got EOF from 127.4.48.194:53903 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:21.706238 20311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:21.694720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:55191)
0811 12:10:21.695218 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:21.695234 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:21.695366 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:21.695722 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:21.695728 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:21.696161 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:21.696344 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.696354 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.697351 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.697356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.698261 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.698273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.698968 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.698974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.699121 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.702885 (+ 3764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:21.702903 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:21.705924 (+ 3021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":263,"threads_started":1}
W20260811 12:10:21.706393 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:21.694788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:55191 (local address 127.4.48.254:43469)
0811 12:10:21.694956 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:21.694962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:21.695314 (+ 352us) server_negotiation.cc:407] Connection header received
0811 12:10:21.695483 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:21.695488 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:21.695555 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:21.695648 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:21.696481 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.697224 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.698394 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.698826 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.698932 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.706151 (+ 7219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:55191: BlockingRecv error: recv got EOF from 127.4.48.195:55191 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:21 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:21.819713 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:21.811927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:47175)
0811 12:10:21.812441 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:21.812463 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:21.812588 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:21.812922 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:21.812925 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:21.813313 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:21.813469 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.813477 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.814457 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.814460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.815379 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:21.815389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.816135 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.816140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:21.816266 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.816816 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:21.816841 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:21.819450 (+ 2609us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":251,"threads_started":1}
W20260811 12:10:21.819972 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:21.811957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:47175 (local address 127.4.48.254:43469)
0811 12:10:21.812139 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:21.812146 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:21.812546 (+ 400us) server_negotiation.cc:407] Connection header received
0811 12:10:21.812678 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:21.812682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:21.812784 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:21.812856 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:21.813605 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.814346 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.815517 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:21.816023 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:21.816120 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:21.819695 (+ 3575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:47175: BlockingRecv error: recv got EOF from 127.4.48.193:47175 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:22.261868 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:22.253671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:54541)
0811 12:10:22.254214 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:22.254234 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:22.254373 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:22.254696 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:22.254701 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:22.255118 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:22.255278 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.255288 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.256338 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.256342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.257241 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.257253 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.257975 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.257981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.258154 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.258735 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:22.258753 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:22.261562 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":275,"threads_started":1}
W20260811 12:10:22.262087 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:22.253748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:54541 (local address 127.4.48.254:43469)
0811 12:10:22.253959 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:22.253965 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:22.254316 (+ 351us) server_negotiation.cc:407] Connection header received
0811 12:10:22.254487 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:22.254491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:22.254552 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:22.254632 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:22.255431 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.256190 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.257381 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.257831 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.257945 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.261843 (+ 3898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:54541: BlockingRecv error: recv got EOF from 127.4.48.194:54541 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:22.715217 20319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:22.707091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:43153)
0811 12:10:22.707592 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:22.707608 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:22.707775 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:22.708094 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:22.708098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:22.708518 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:22.708678 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.708688 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.709706 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.709710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.710765 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.710783 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.711690 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.711697 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.711841 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.712444 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:22.712462 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:22.714989 (+ 2527us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":273,"threads_started":1}
W20260811 12:10:22.715422 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:22.707168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:43153 (local address 127.4.48.254:43469)
0811 12:10:22.707355 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:22.707361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:22.707714 (+ 353us) server_negotiation.cc:407] Connection header received
0811 12:10:22.707901 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:22.707906 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:22.707969 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:22.708050 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:22.708814 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.709568 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.710919 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.711534 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.711656 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.715198 (+ 3542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:43153: BlockingRecv error: recv got EOF from 127.4.48.195:43153 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:22 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:22.829834 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:22.820727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:34849)
0811 12:10:22.821285 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:22.821303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:22.821442 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:22.821829 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:22.821835 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:22.822260 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:22.822444 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.822455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.823536 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.823541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.824564 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:22.824577 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.825470 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.825477 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:22.825639 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.826278 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:22.826299 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:22.829437 (+ 3138us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":272,"threads_started":1}
W20260811 12:10:22.830076 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:22.820851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:34849 (local address 127.4.48.254:43469)
0811 12:10:22.821068 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:22.821075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:22.821384 (+ 309us) server_negotiation.cc:407] Connection header received
0811 12:10:22.821556 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:22.821562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:22.821670 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:22.821775 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:22.822591 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.823387 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.824751 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:22.825277 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:22.825406 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:22.829673 (+ 4267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:34849: BlockingRecv error: recv got EOF from 127.4.48.193:34849 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:23.270576 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:23.262774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:49843)
0811 12:10:23.263351 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:23.263371 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:23.263483 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:23.263753 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:23.263756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:23.264138 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:23.264299 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.264307 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.265388 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.265392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.266295 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.266306 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.266978 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.266982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.267093 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.267589 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:23.267605 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:23.270311 (+ 2706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":301,"threads_started":1}
W20260811 12:10:23.270750 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:23.262905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:49843 (local address 127.4.48.254:43469)
0811 12:10:23.263076 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:23.263080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:23.263435 (+ 355us) server_negotiation.cc:407] Connection header received
0811 12:10:23.263567 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:23.263571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:23.263634 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:23.263739 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:23.264435 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.265247 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.266428 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.266840 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.266930 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.270537 (+ 3607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:49843: BlockingRecv error: recv got EOF from 127.4.48.194:49843 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:23.723644 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:23.716032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:58483)
0811 12:10:23.716545 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:23.716561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:23.716673 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:23.716935 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:23.716938 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:23.717306 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:23.717469 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.717478 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.718390 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.718393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.719337 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.719350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.720060 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.720065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.720188 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.720721 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:23.720738 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:23.723407 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":263,"threads_started":1}
W20260811 12:10:23.723861 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:23.716093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:58483 (local address 127.4.48.254:43469)
0811 12:10:23.716246 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:23.716250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:23.716621 (+ 371us) server_negotiation.cc:407] Connection header received
0811 12:10:23.716771 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:23.716774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:23.716833 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:23.716909 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:23.717574 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.718284 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.719463 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.719911 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.720005 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.723587 (+ 3582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:58483: BlockingRecv error: recv got EOF from 127.4.48.195:58483 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:23 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:23.838205 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:23.830665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:47561)
0811 12:10:23.831099 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:23.831115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:23.831228 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:23.831499 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:23.831502 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:23.831889 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:23.832069 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.832079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.833025 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.833028 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.833930 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:23.833940 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.834516 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.834520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:23.834635 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.835173 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:23.835189 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:23.837953 (+ 2764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":227,"threads_started":1}
W20260811 12:10:23.838375 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:23.830729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:47561 (local address 127.4.48.254:43469)
0811 12:10:23.830992 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:23.830997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:23.831182 (+ 185us) server_negotiation.cc:407] Connection header received
0811 12:10:23.831314 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:23.831317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:23.831396 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:23.831474 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:23.832171 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.832918 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.834027 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:23.834418 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:23.834501 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:23.838145 (+ 3644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:47561: BlockingRecv error: recv got EOF from 127.4.48.193:47561 (error 108)
Metrics: {"server-negotiator.queue_time_us":139}
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:24.279404 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:24.271366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:43671)
0811 12:10:24.271842 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:24.271863 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:24.271990 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:24.272293 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:24.272297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:24.272681 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:24.272879 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.272890 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.273901 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.273905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.274802 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.274814 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.275522 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.275528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.275709 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.276234 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:24.276250 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:24.279066 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":238,"threads_started":1}
W20260811 12:10:24.279476 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:24.271450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:43671 (local address 127.4.48.254:43469)
0811 12:10:24.271630 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:24.271637 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:24.271943 (+ 306us) server_negotiation.cc:407] Connection header received
0811 12:10:24.272101 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:24.272105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:24.272166 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:24.272246 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:24.273031 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.273752 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.274953 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.275386 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.275496 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.279279 (+ 3783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:43671: BlockingRecv error: recv got EOF from 127.4.48.194:43671 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.195@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:24.732609 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:24.724500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.195:44789)
0811 12:10:24.724958 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:24.724978 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:24.725107 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:24.725417 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:24.725425 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:24.725777 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:24.725962 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.725971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.726995 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.726999 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.727958 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.727972 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.728865 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.728869 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.729033 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.729629 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:24.729646 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:24.732275 (+ 2629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
W20260811 12:10:24.732738 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:24.724579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:44789 (local address 127.4.48.254:43469)
0811 12:10:24.724743 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:24.724747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:24.725052 (+ 305us) server_negotiation.cc:407] Connection header received
0811 12:10:24.725227 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:24.725230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:24.725294 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:24.725376 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:24.726085 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.726859 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.728084 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.728720 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.728846 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.732523 (+ 3677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.195:44789: BlockingRecv error: recv got EOF from 127.4.48.195:44789 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:24 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.193@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:24.847131 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:24.839250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.193:38295)
0811 12:10:24.839758 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:24.839775 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:24.839883 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:24.840148 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:24.840151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:24.840562 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:24.840775 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.840784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.841677 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.841681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.842573 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:24.842584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.843475 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.843480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:24.843631 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.844285 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:24.844310 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:24.846883 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":265,"threads_started":1}
W20260811 12:10:24.847258 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:24.839307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:38295 (local address 127.4.48.254:43469)
0811 12:10:24.839491 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:24.839498 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:24.839854 (+ 356us) server_negotiation.cc:407] Connection header received
0811 12:10:24.839963 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:24.839967 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:24.840044 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:24.840121 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:24.840893 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.841577 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.842681 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:24.843321 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:24.843439 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:24.847058 (+ 3619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.193:38295: BlockingRecv error: recv got EOF from 127.4.48.193:38295 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 12:10:25 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
Aug 11 12:10:25 dist-test-slave-nj21 krb5kdc[18521](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.4.48.194@KRBTEST.COM for oryx/127.4.48.254@KRBTEST.COM, Server not found in Kerberos database
I20260811 12:10:25.288882 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:25.280431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:43469 (local address 127.4.48.194:56909)
0811 12:10:25.280960 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:25.280983 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:25.281134 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:25.281489 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:25.281494 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:25.281952 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:25.282166 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:25.282177 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.283347 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.283353 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:25.284621 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:25.284638 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.285329 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.285333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:25.285455 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:25.285966 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:25.285982 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:25.288619 (+ 2637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.4.48.254:43469: Server oryx/127.4.48.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":293,"threads_started":1}
W20260811 12:10:25.289018 19804 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:25.280507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:56909 (local address 127.4.48.254:43469)
0811 12:10:25.280709 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:25.280717 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:25.281073 (+ 356us) server_negotiation.cc:407] Connection header received
0811 12:10:25.281268 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:25.281274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:25.281343 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:25.281428 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:25.282320 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.283185 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.284760 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.285212 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.285321 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:25.288818 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.4.48.194:56909: BlockingRecv error: recv got EOF from 127.4.48.194:56909 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:10:25.571465 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19533
I20260811 12:10:25.582633 20333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:25.572428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51750 (local address 127.4.48.193:43347)
0811 12:10:25.572944 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:25.572950 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:25.572976 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:10:25.573031 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:25.573038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:25.573250 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:25.573396 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:25.574664 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.575626 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.576745 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.577294 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.577393 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:25.579927 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:25.579936 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:25.579949 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:25.579980 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:25.582161 (+ 2181us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:10:25.582391 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51750: Request ticket server kudu/127.4.48.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":223,"threads_started":1}
W20260811 12:10:25.582731 20333 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51750: Request ticket server kudu/127.4.48.193@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:10:25.582743 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19533: Not authorized: Client connection negotiation failed: client connection to 127.4.48.193:43347: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:10:25.582985 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 19533
I20260811 12:10:25.613190 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19806
I20260811 12:10:25.624567 20334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:25.614178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.4.48.194:43115)
0811 12:10:25.614769 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:25.614777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:25.614810 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:10:25.614875 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:25.614882 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:25.615091 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:25.615269 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:25.616473 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.617528 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.618661 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.619223 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.619325 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:25.621601 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:25.621611 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:25.621625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:25.621661 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:25.624093 (+ 2432us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:10:25.624341 (+ 248us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43082: Request ticket server kudu/127.4.48.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":248,"threads_started":1}
W20260811 12:10:25.624673 20334 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43082: Request ticket server kudu/127.4.48.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:10:25.624696 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19806: Not authorized: Client connection negotiation failed: client connection to 127.4.48.194:43115: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:10:25.624871 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 19806
I20260811 12:10:25.649809 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20029
I20260811 12:10:25.660917 20335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:25.650758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.4.48.195:44485)
0811 12:10:25.651289 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:25.651295 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:25.651322 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:10:25.651390 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:25.651397 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:25.651615 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:25.651800 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:25.653034 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.654049 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.655144 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:25.655798 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:25.655887 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:25.658217 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:25.658224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:25.658237 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:25.658266 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:25.660454 (+ 2188us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 12:10:25.660685 (+ 231us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37458: Request ticket server kudu/127.4.48.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
W20260811 12:10:25.661021 20335 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37458: Request ticket server kudu/127.4.48.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 12:10:25.661078 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20029: Not authorized: Client connection negotiation failed: client connection to 127.4.48.195:44485: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.4.48.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 12:10:25.661255 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 20029
I20260811 12:10:25.684592 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 19423
I20260811 12:10:25.685863 19524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:25.800485 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 19423
2026-08-11T12:10:25Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21167 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T12:10:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:25Z Disabled control of system clock
I20260811 12:10:25.886663 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39579
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39579
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 12:10:26.211726 20344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:26.212003 20344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:26.212121 20344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:26.219058 20344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:26.219151 20344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:26.219211 20344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:26.219290 20344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:26.219352 20344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:26.228037 20344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39579
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:26.229286 20344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:26.231253 20344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:26.242368 20355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:26.242456 20352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:26.242457 20357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:26.243129 20344 server_base.cc:1034] running on GCE node
I20260811 12:10:26.243870 20344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:26.245172 20344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:26.246358 20344 hybrid_clock.cc:648] HybridClock initialized: now 1786450226246308 us; error 36 us; skew 500 ppm
I20260811 12:10:26.246691 20344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:26.248847 20344 webserver.cc:460] Webserver started at http://127.4.48.254:34651/ using document root <none> and password file <none>
I20260811 12:10:26.249509 20344 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:26.249601 20344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:26.249917 20344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:26.252789 20344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "374138f6f24a42ad9d01d0d45c955406"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "95d16066e19f354c8bf68cc994c90801"
server_key_iv: "ed2812a1237fca94caabfe92223132d7"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.253545 20344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "374138f6f24a42ad9d01d0d45c955406"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "95d16066e19f354c8bf68cc994c90801"
server_key_iv: "ed2812a1237fca94caabfe92223132d7"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.258450 20344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 12:10:26.261586 20372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:26.262791 20344 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:10:26.262949 20344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "374138f6f24a42ad9d01d0d45c955406"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "95d16066e19f354c8bf68cc994c90801"
server_key_iv: "ed2812a1237fca94caabfe92223132d7"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.263151 20344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:26.278959 20344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:26.279721 20344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:26.279980 20344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:26.295831 20344 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39579
I20260811 12:10:26.295853 20460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39579 every 8 connection(s)
I20260811 12:10:26.297091 20344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:10:26.298365 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 20344
I20260811 12:10:26.298553 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:10:26.299399 4291 external_mini_cluster.cc:1428] Setting key bffb4a4ccbb51f66a1dca6e3bee3222b
I20260811 12:10:26.301828 20461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:26.312822 20469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:26.301477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49182 (local address 127.4.48.254:39579)
0811 12:10:26.302400 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:26.302411 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:26.302453 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:10:26.304064 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:26.304086 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:26.304528 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:26.304832 (+ 304us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:26.305497 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.306803 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.308683 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.309304 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.309436 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:26.310660 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:26.310668 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:26.310679 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:26.311195 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:26.311269 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:26.311292 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:26.311388 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:26.311601 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 12:10:26.311937 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":335,"threads_started":1}
I20260811 12:10:26.314091 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406: Bootstrap starting.
I20260811 12:10:26.317178 20461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:26.318127 20461 log.cc:824] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:26.320900 20461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406: No bootstrap required, opened a new log
I20260811 12:10:26.327955 20461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } }
I20260811 12:10:26.328274 20461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:26.328367 20461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 374138f6f24a42ad9d01d0d45c955406, State: Initialized, Role: FOLLOWER
I20260811 12:10:26.328922 20461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } }
I20260811 12:10:26.329139 20461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:10:26.329263 20461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:10:26.329423 20461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:26.331413 20461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } }
I20260811 12:10:26.331933 20461 leader_election.cc:302] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 374138f6f24a42ad9d01d0d45c955406; no voters:
I20260811 12:10:26.332487 20461 leader_election.cc:288] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:10:26.332619 20472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:26.333282 20472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [term 1 LEADER]: Becoming Leader. State: Replica: 374138f6f24a42ad9d01d0d45c955406, State: Running, Role: LEADER
I20260811 12:10:26.333735 20472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } }
I20260811 12:10:26.334347 20461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:10:26.336381 20477 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 374138f6f24a42ad9d01d0d45c955406. Latest consensus state: current_term: 1 leader_uuid: "374138f6f24a42ad9d01d0d45c955406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } } }
I20260811 12:10:26.336459 20474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "374138f6f24a42ad9d01d0d45c955406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "374138f6f24a42ad9d01d0d45c955406" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39579 } } }
I20260811 12:10:26.336781 20477 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:26.336787 20474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:26.337863 20492 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:10:26.342391 20492 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:10:26.350178 20492 catalog_manager.cc:1347] Generated new cluster ID: 20b3a03dbda64d18a2995f29e03d28f2
I20260811 12:10:26.350282 20492 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:10:26.389199 20492 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:10:26.390992 20492 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:10:26.400234 20492 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 374138f6f24a42ad9d01d0d45c955406: Generated new TSK 0
I20260811 12:10:26.401213 20492 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:10:26.409327 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--builtin_ntp_servers=127.4.48.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 12:10:26.736379 20510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:26.736630 20510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:26.736711 20510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:26.743651 20510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:26.743752 20510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:26.743849 20510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:10:26.752537 20510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:26.753664 20510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:26.755330 20510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:26.767912 20518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:26.767963 20522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:26.767982 20519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:26.768874 20510 server_base.cc:1034] running on GCE node
I20260811 12:10:26.769399 20510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:26.770246 20510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:26.771447 20510 hybrid_clock.cc:648] HybridClock initialized: now 1786450226771393 us; error 45 us; skew 500 ppm
I20260811 12:10:26.771778 20510 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:26.779081 20510 webserver.cc:460] Webserver started at http://127.4.48.193:44253/ using document root <none> and password file <none>
I20260811 12:10:26.779808 20510 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:26.779915 20510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:26.780251 20510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:26.782953 20510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "5fdd338b688847fdbde3eb42936ce8df"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "790fb02b950d2f8c78b66e281dd2f2b5"
server_key_iv: "c6a73c2eb870a290b68644f1f9534509"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.783829 20510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "5fdd338b688847fdbde3eb42936ce8df"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "790fb02b950d2f8c78b66e281dd2f2b5"
server_key_iv: "c6a73c2eb870a290b68644f1f9534509"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.788568 20510 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:10:26.791805 20529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:26.793210 20510 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:10:26.793375 20510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "5fdd338b688847fdbde3eb42936ce8df"
format_stamp: "Formatted at 2026-08-11 12:10:26 on dist-test-slave-nj21"
server_key: "790fb02b950d2f8c78b66e281dd2f2b5"
server_key_iv: "c6a73c2eb870a290b68644f1f9534509"
server_key_version: "encryptionkey@0"
I20260811 12:10:26.793567 20510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:26.806689 20510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:26.807433 20510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:26.807732 20510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:26.808588 20510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:26.809962 20510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:26.810055 20510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:26.810163 20510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:26.810237 20510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:26.839897 20510 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:39893
I20260811 12:10:26.839951 20732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:39893 every 8 connection(s)
I20260811 12:10:26.841128 20510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:10:26.845005 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 20510
I20260811 12:10:26.845193 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:10:26.845851 4291 external_mini_cluster.cc:1428] Setting key 53259a01bf2705a6529c440237f8d89f
I20260811 12:10:26.848640 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--builtin_ntp_servers=127.4.48.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:10:26.853554 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:26.843468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56271 (local address 127.4.48.254:39579)
0811 12:10:26.843881 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:26.843885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:26.845138 (+ 1253us) server_negotiation.cc:407] Connection header received
0811 12:10:26.847053 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:26.847059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:26.847129 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:26.847234 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:26.848292 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.849243 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.850770 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.851328 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.851443 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:26.852787 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:26.852794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:26.852797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:26.852816 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:26.852863 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:26.852878 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:26.852961 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:26.853157 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:10:26.853248 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":216,"threads_started":1}
I20260811 12:10:26.854610 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:26.844045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39579 (local address 127.4.48.193:56271)
0811 12:10:26.845000 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:26.845035 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:26.846816 (+ 1781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:26.847429 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:26.847437 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:26.847518 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:26.848120 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:26.848138 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.849387 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.849391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:26.850626 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:26.850639 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:26.851518 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:26.851524 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:26.851795 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:26.852469 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:26.852472 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:26.852515 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:26.852593 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:26.852963 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:26.852970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:26.852976 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:10:26.853116 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 12:10:26.853438 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":296,"threads_started":1}
I20260811 12:10:26.856058 20733 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39579
I20260811 12:10:26.856364 20733 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:26.857193 20733 heartbeater.cc:511] Master 127.4.48.254:39579 requested a full tablet report, sending...
I20260811 12:10:26.859292 20399 ts_manager.cc:194] Registered new tserver with Master: 5fdd338b688847fdbde3eb42936ce8df (127.4.48.193:39893)
I20260811 12:10:26.860836 20399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.193:56271
W20260811 12:10:27.174865 20743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:27.175113 20743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:27.175215 20743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:27.182123 20743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:27.182206 20743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:27.182337 20743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:10:27.190941 20743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:27.192117 20743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:27.193814 20743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:27.206012 20750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:27.205966 20753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:27.206055 20751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:27.206779 20743 server_base.cc:1034] running on GCE node
I20260811 12:10:27.207309 20743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:27.208186 20743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:27.209393 20743 hybrid_clock.cc:648] HybridClock initialized: now 1786450227209329 us; error 52 us; skew 500 ppm
I20260811 12:10:27.209682 20743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:27.211812 20743 webserver.cc:460] Webserver started at http://127.4.48.194:33979/ using document root <none> and password file <none>
I20260811 12:10:27.212443 20743 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:27.212541 20743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:27.212857 20743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:27.215524 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "4d5ff873481642eeb6d5c0de7bb7b37f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "692966b376ee6190dcdaa413906adf37"
server_key_iv: "b18ba8a02877a5ad087dccd2705e82ba"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.216311 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "4d5ff873481642eeb6d5c0de7bb7b37f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "692966b376ee6190dcdaa413906adf37"
server_key_iv: "b18ba8a02877a5ad087dccd2705e82ba"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.220973 20743 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:10:27.224085 20769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.225320 20743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 12:10:27.225464 20743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "4d5ff873481642eeb6d5c0de7bb7b37f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "692966b376ee6190dcdaa413906adf37"
server_key_iv: "b18ba8a02877a5ad087dccd2705e82ba"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.225644 20743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:27.244928 20743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:27.245635 20743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:27.245905 20743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:27.246802 20743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:27.248183 20743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:27.248274 20743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.248373 20743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:27.248440 20743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.277652 20743 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:41521
I20260811 12:10:27.277691 20981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:41521 every 8 connection(s)
I20260811 12:10:27.278846 20743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:10:27.285663 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 20743
I20260811 12:10:27.285815 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:10:27.286259 4291 external_mini_cluster.cc:1428] Setting key 43034c995cc44bbaf6f08e39ba40f51d
I20260811 12:10:27.289120 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--builtin_ntp_servers=127.4.48.212:45879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 12:10:27.291285 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.281238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:57545 (local address 127.4.48.254:39579)
0811 12:10:27.281410 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.281415 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.282714 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 12:10:27.284441 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.284447 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.284527 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.284616 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.285662 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.286498 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.288626 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.289031 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.289145 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.290505 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:27.290511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:27.290514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:27.290531 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:27.290562 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:27.290573 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:27.290649 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:27.290869 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 12:10:27.290968 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:10:27.292411 20986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.281694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39579 (local address 127.4.48.194:57545)
0811 12:10:27.282559 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:27.282601 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:27.284167 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:27.284848 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:27.284856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:27.284936 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:27.285498 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:27.285515 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.286653 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.286658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:27.288485 (+ 1827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:27.288498 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.289259 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.289267 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:27.289511 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.290173 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:27.290175 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:27.290217 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:27.290313 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:27.290660 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:27.290672 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:27.290681 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:10:27.290839 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 12:10:27.291265 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":258,"threads_started":1}
I20260811 12:10:27.293735 20983 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39579
I20260811 12:10:27.294025 20983 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:27.294864 20983 heartbeater.cc:511] Master 127.4.48.254:39579 requested a full tablet report, sending...
I20260811 12:10:27.300581 20399 ts_manager.cc:194] Registered new tserver with Master: 4d5ff873481642eeb6d5c0de7bb7b37f (127.4.48.194:41521)
I20260811 12:10:27.301643 20399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.194:57545
W20260811 12:10:27.616238 20989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:27.616501 20989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:27.616629 20989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:27.623512 20989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:27.623606 20989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:27.623755 20989 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:27.632473 20989 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:45879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39579
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:27.633615 20989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:27.635326 20989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:27.647464 21002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:27.647563 20998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:27.647902 20997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:27.648169 20989 server_base.cc:1034] running on GCE node
I20260811 12:10:27.648712 20989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:27.649554 20989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:27.650735 20989 hybrid_clock.cc:648] HybridClock initialized: now 1786450227650690 us; error 36 us; skew 500 ppm
I20260811 12:10:27.651010 20989 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:27.652973 20989 webserver.cc:460] Webserver started at http://127.4.48.195:42071/ using document root <none> and password file <none>
I20260811 12:10:27.653617 20989 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:27.653714 20989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:27.654006 20989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:27.656632 20989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "4f52bffab8f94a3cb59371f91256978f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "9b28dc96c27a063532bbbc3e85dc26aa"
server_key_iv: "2d6278a97fc8a6b9b5eec2bacd4e46fa"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.657366 20989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "4f52bffab8f94a3cb59371f91256978f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "9b28dc96c27a063532bbbc3e85dc26aa"
server_key_iv: "2d6278a97fc8a6b9b5eec2bacd4e46fa"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.662000 20989 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 12:10:27.665159 21018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.666457 20989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:10:27.666601 20989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "4f52bffab8f94a3cb59371f91256978f"
format_stamp: "Formatted at 2026-08-11 12:10:27 on dist-test-slave-nj21"
server_key: "9b28dc96c27a063532bbbc3e85dc26aa"
server_key_iv: "2d6278a97fc8a6b9b5eec2bacd4e46fa"
server_key_version: "encryptionkey@0"
I20260811 12:10:27.666781 20989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:27.697800 20989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:27.698575 20989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:27.698846 20989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:27.699747 20989 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:27.701116 20989 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:27.701210 20989 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.701321 20989 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:27.701390 20989 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:27.730822 20989 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39091
I20260811 12:10:27.730887 21225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39091 every 8 connection(s)
I20260811 12:10:27.732066 20989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:27.736950 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 20989
I20260811 12:10:27.737133 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:10:27.737677 4291 external_mini_cluster.cc:1428] Setting key b102f6bce8502c1f18919614aff60c80
I20260811 12:10:27.744877 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.734568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:33565 (local address 127.4.48.254:39579)
0811 12:10:27.734787 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.734791 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.736315 (+ 1524us) server_negotiation.cc:407] Connection header received
0811 12:10:27.738652 (+ 2337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.738657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.738739 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.738846 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.740215 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.741146 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.742593 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.742991 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.743078 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.744250 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:27.744255 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:27.744258 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:27.744275 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:27.744307 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:27.744327 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:27.744396 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:27.744560 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 12:10:27.744638 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 12:10:27.745802 21230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.735173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39579 (local address 127.4.48.195:33565)
0811 12:10:27.736152 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:27.736188 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:27.738366 (+ 2178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:27.739086 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:27.739097 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:27.739212 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:27.740044 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:27.740070 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.741282 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.741285 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:27.742491 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:27.742503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.743152 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.743156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:27.743351 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.743963 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:27.743966 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:27.744011 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:27.744084 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:27.744399 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:27.744405 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:27.744410 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:10:27.744533 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 12:10:27.744804 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":303,"threads_started":1}
I20260811 12:10:27.747031 21226 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39579
I20260811 12:10:27.747309 21226 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:27.748119 21226 heartbeater.cc:511] Master 127.4.48.254:39579 requested a full tablet report, sending...
I20260811 12:10:27.749459 20399 ts_manager.cc:194] Registered new tserver with Master: 4f52bffab8f94a3cb59371f91256978f (127.4.48.195:39091)
I20260811 12:10:27.750206 20399 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.195:33565
I20260811 12:10:27.752772 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:10:27.758562 20735 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:27.757115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49186 (local address 127.4.48.254:39579)
0811 12:10:27.757312 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.757318 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.757669 (+ 351us) server_negotiation.cc:407] Connection header received
0811 12:10:27.757823 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.757828 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.757906 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.757986 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.758378 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49186: BlockingRecv error: recv got EOF from 127.0.0.1:49186 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:10:27.759343 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20510
I20260811 12:10:27.766155 21243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.760163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51646 (local address 127.4.48.193:39893)
0811 12:10:27.760608 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.760614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.760639 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:10:27.760689 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.760695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.760867 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.760979 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.761261 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.762185 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.763314 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.763916 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.764014 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.764907 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:27.764914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:27.764925 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:27.765465 (+ 540us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:27.765507 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:27.765525 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:27.765614 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:27.765686 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 12:10:27.765789 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":212,"threads_started":1}
I20260811 12:10:27.767313 20726 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:27.846731 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 20510
I20260811 12:10:27.867488 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20743
I20260811 12:10:27.875022 21247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.868355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.4.48.194:41521)
0811 12:10:27.868907 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.868914 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.868943 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:10:27.869006 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.869012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.869232 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.869371 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.869711 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.870748 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.872106 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.872686 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.872787 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.873660 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:27.873669 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:27.873684 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:27.874288 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:27.874349 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:27.874369 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:27.874456 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:27.874590 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:10:27.874712 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":285,"threads_started":1}
I20260811 12:10:27.876341 20969 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:27.955153 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 20743
I20260811 12:10:27.978407 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20989
I20260811 12:10:27.986446 21249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:27.979322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.4.48.195:39091)
0811 12:10:27.979982 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:27.979989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:27.980022 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:10:27.980084 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:27.980089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:27.980314 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:27.980472 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:27.980950 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.981990 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.983238 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:27.983939 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:27.984054 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:27.985039 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:27.985048 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:27.985062 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:27.985626 (+ 564us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:27.985682 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:27.985701 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:27.985803 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:27.985959 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 12:10:27.986095 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":353,"threads_started":1}
I20260811 12:10:27.987617 21215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:28.067772 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 20989
I20260811 12:10:28.090579 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 20344
I20260811 12:10:28.091840 20450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:28.192936 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 20344
2026-08-11T12:10:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2387 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T12:10:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:28Z Disabled control of system clock
I20260811 12:10:28.275635 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46733
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:10:28.603915 21258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:28.604229 21258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:28.604333 21258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:28.611303 21258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:28.611404 21258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:28.611449 21258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:28.611510 21258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:28.611552 21258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:28.620428 21258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46733
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:28.621616 21258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:28.623319 21258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:28.634344 21269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:28.634404 21265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:28.634598 21264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:28.635409 21258 server_base.cc:1034] running on GCE node
I20260811 12:10:28.636178 21258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:28.637456 21258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:28.638664 21258 hybrid_clock.cc:648] HybridClock initialized: now 1786450228638617 us; error 40 us; skew 500 ppm
I20260811 12:10:28.638981 21258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:28.641146 21258 webserver.cc:460] Webserver started at http://127.4.48.254:44245/ using document root <none> and password file <none>
I20260811 12:10:28.641795 21258 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:28.641901 21258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:28.642205 21258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:28.645047 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "653a46964a97434f86bb1a31d21ea762"
format_stamp: "Formatted at 2026-08-11 12:10:28 on dist-test-slave-nj21"
server_key: "3535a6d272f83b1bfe2d44f695423096"
server_key_iv: "48a8ef7af57508da1302b74bd552cf2f"
server_key_version: "encryptionkey@0"
I20260811 12:10:28.645798 21258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "653a46964a97434f86bb1a31d21ea762"
format_stamp: "Formatted at 2026-08-11 12:10:28 on dist-test-slave-nj21"
server_key: "3535a6d272f83b1bfe2d44f695423096"
server_key_iv: "48a8ef7af57508da1302b74bd552cf2f"
server_key_version: "encryptionkey@0"
I20260811 12:10:28.650902 21258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:10:28.654201 21278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:28.655485 21258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:28.655627 21258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "653a46964a97434f86bb1a31d21ea762"
format_stamp: "Formatted at 2026-08-11 12:10:28 on dist-test-slave-nj21"
server_key: "3535a6d272f83b1bfe2d44f695423096"
server_key_iv: "48a8ef7af57508da1302b74bd552cf2f"
server_key_version: "encryptionkey@0"
I20260811 12:10:28.655858 21258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:28.661195 21258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:28.661892 21258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:28.662137 21258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:28.679481 21258 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:46733
I20260811 12:10:28.679510 21372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:46733 every 8 connection(s)
I20260811 12:10:28.680761 21258 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:10:28.684423 21375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:28.685534 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 21258
I20260811 12:10:28.685716 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:10:28.686360 4291 external_mini_cluster.cc:1428] Setting key 1f1f8cf858d21131d4076edcbf681abc
I20260811 12:10:28.694981 21379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:28.688280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43664 (local address 127.4.48.254:46733)
0811 12:10:28.689427 (+ 1147us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:28.689456 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:28.689509 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:10:28.691084 (+ 1575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:28.691111 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:28.691654 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:28.692098 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:28.693046 (+ 948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:28.693054 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:28.693068 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:28.693086 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:28.693162 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:28.693184 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:28.693416 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:28.693534 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 12:10:28.693788 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":368,"threads_started":1}
I20260811 12:10:28.698043 21375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762: Bootstrap starting.
I20260811 12:10:28.700865 21375 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:28.701792 21375 log.cc:824] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:28.704394 21375 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762: No bootstrap required, opened a new log
I20260811 12:10:28.711567 21375 raft_consensus.cc:348] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } }
I20260811 12:10:28.712038 21375 raft_consensus.cc:374] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:28.712121 21375 raft_consensus.cc:729] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 653a46964a97434f86bb1a31d21ea762, State: Initialized, Role: FOLLOWER
I20260811 12:10:28.712637 21375 consensus_queue.cc:260] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } }
I20260811 12:10:28.712843 21375 raft_consensus.cc:388] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:10:28.712949 21375 raft_consensus.cc:482] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:10:28.713073 21375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:28.715140 21375 raft_consensus.cc:504] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } }
I20260811 12:10:28.715577 21375 leader_election.cc:302] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 653a46964a97434f86bb1a31d21ea762; no voters:
I20260811 12:10:28.716182 21375 leader_election.cc:288] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:10:28.716338 21381 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:28.717027 21381 raft_consensus.cc:686] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [term 1 LEADER]: Becoming Leader. State: Replica: 653a46964a97434f86bb1a31d21ea762, State: Running, Role: LEADER
I20260811 12:10:28.717540 21381 consensus_queue.cc:237] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } }
I20260811 12:10:28.718400 21375 sys_catalog.cc:564] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:10:28.720381 21385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 653a46964a97434f86bb1a31d21ea762. Latest consensus state: current_term: 1 leader_uuid: "653a46964a97434f86bb1a31d21ea762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } } }
I20260811 12:10:28.720760 21385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:28.721489 21383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "653a46964a97434f86bb1a31d21ea762" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "653a46964a97434f86bb1a31d21ea762" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46733 } } }
I20260811 12:10:28.721807 21383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:28.725023 21392 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:10:28.730109 21392 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:10:28.739151 21392 catalog_manager.cc:1347] Generated new cluster ID: 7745cfb3591c4264a0d88b65d920b862
I20260811 12:10:28.739281 21392 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:10:28.756269 21392 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:10:28.756711 21392 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:10:28.767436 21392 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 653a46964a97434f86bb1a31d21ea762: Generated new TSK 0
I20260811 12:10:28.768280 21392 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:10:28.777961 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--builtin_ntp_servers=127.4.48.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 12:10:29.112444 21413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:29.112711 21413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:29.112799 21413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:29.120026 21413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:29.120116 21413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:29.120232 21413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:10:29.129132 21413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:29.130360 21413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:29.132153 21413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:29.144932 21425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:29.144999 21423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:29.144884 21428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:29.145622 21413 server_base.cc:1034] running on GCE node
I20260811 12:10:29.146219 21413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:29.147172 21413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:29.148391 21413 hybrid_clock.cc:648] HybridClock initialized: now 1786450229148336 us; error 40 us; skew 500 ppm
I20260811 12:10:29.148698 21413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:29.150898 21413 webserver.cc:460] Webserver started at http://127.4.48.193:44593/ using document root <none> and password file <none>
I20260811 12:10:29.151559 21413 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:29.151659 21413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:29.152000 21413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:29.154922 21413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "651f2b59b89844fab952fbcf543bb42a"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "431a46411f806807846c0fd39bb4f458"
server_key_iv: "72288cb99fb1bb496903e6c293ee53ad"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.155665 21413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "651f2b59b89844fab952fbcf543bb42a"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "431a46411f806807846c0fd39bb4f458"
server_key_iv: "72288cb99fb1bb496903e6c293ee53ad"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.160653 21413 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 12:10:29.163920 21439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.165148 21413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:10:29.165297 21413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "651f2b59b89844fab952fbcf543bb42a"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "431a46411f806807846c0fd39bb4f458"
server_key_iv: "72288cb99fb1bb496903e6c293ee53ad"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.165486 21413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:29.170852 21413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:29.171571 21413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:29.171880 21413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:29.172732 21413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:29.174036 21413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:29.174129 21413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.174233 21413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:29.174302 21413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.210040 21413 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:40541
I20260811 12:10:29.210093 21630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:40541 every 8 connection(s)
I20260811 12:10:29.211310 21413 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:10:29.218876 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 21413
I20260811 12:10:29.219072 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:10:29.219065 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:29.213799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:53625 (local address 127.4.48.254:46733)
0811 12:10:29.214207 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:29.214212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:29.215394 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 12:10:29.217027 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:29.217031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:29.217085 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:29.217179 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:29.218270 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:29.218274 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:29.218277 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:29.218293 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:29.218338 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:29.218350 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:29.218481 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:29.218695 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:10:29.218763 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":200,"threads_started":1}
I20260811 12:10:29.219830 4291 external_mini_cluster.cc:1428] Setting key 69306c6b35aa422dae4625f9b19ede72
I20260811 12:10:29.220044 21640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:29.214350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46733 (local address 127.4.48.193:53625)
0811 12:10:29.215246 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:29.215284 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:29.216807 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:29.217351 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:29.217358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:29.217442 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:29.218005 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:29.218007 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:29.218043 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:29.218124 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:29.218483 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:29.218496 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:29.218504 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:10:29.218766 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 12:10:29.219010 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":271,"threads_started":1}
I20260811 12:10:29.221565 21635 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46733
I20260811 12:10:29.221858 21635 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:29.222581 21635 heartbeater.cc:511] Master 127.4.48.254:46733 requested a full tablet report, sending...
I20260811 12:10:29.223367 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--builtin_ntp_servers=127.4.48.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:10:29.224699 21303 ts_manager.cc:194] Registered new tserver with Master: 651f2b59b89844fab952fbcf543bb42a (127.4.48.193:40541)
W20260811 12:10:29.553884 21641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:29.554147 21641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:29.554241 21641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:29.561260 21641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:29.561353 21641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:29.561462 21641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:10:29.570184 21641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:29.571352 21641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:29.573102 21641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:29.585661 21657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:29.585661 21652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:29.585702 21653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:29.586964 21641 server_base.cc:1034] running on GCE node
I20260811 12:10:29.587580 21641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:29.588640 21641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:29.589870 21641 hybrid_clock.cc:648] HybridClock initialized: now 1786450229589823 us; error 44 us; skew 500 ppm
I20260811 12:10:29.590193 21641 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:29.592437 21641 webserver.cc:460] Webserver started at http://127.4.48.194:40319/ using document root <none> and password file <none>
I20260811 12:10:29.593122 21641 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:29.593225 21641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:29.593547 21641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:29.596458 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "cd7ae48eb74e4ea5a36e8bd19c8046cf"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "2308f5b1c264548c7d7ff6e7190d65a8"
server_key_iv: "714e890540e7ac57ec9d172c265badea"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.597440 21641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "cd7ae48eb74e4ea5a36e8bd19c8046cf"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "2308f5b1c264548c7d7ff6e7190d65a8"
server_key_iv: "714e890540e7ac57ec9d172c265badea"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.602469 21641 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:10:29.605793 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.607023 21641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:10:29.607177 21641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "cd7ae48eb74e4ea5a36e8bd19c8046cf"
format_stamp: "Formatted at 2026-08-11 12:10:29 on dist-test-slave-nj21"
server_key: "2308f5b1c264548c7d7ff6e7190d65a8"
server_key_iv: "714e890540e7ac57ec9d172c265badea"
server_key_version: "encryptionkey@0"
I20260811 12:10:29.607372 21641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:29.613211 21641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:29.613960 21641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:29.614200 21641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:29.615082 21641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:29.616461 21641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:29.616556 21641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.616660 21641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:29.616729 21641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:29.646883 21641 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:38845
I20260811 12:10:29.646934 21884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:38845 every 8 connection(s)
I20260811 12:10:29.648183 21641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:10:29.649835 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 21641
I20260811 12:10:29.649999 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:10:29.650418 4291 external_mini_cluster.cc:1428] Setting key 0922df9be84e7ea65755dccd33274f82
I20260811 12:10:29.653339 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--builtin_ntp_servers=127.4.48.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 12:10:29.656256 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:29.650652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:43039 (local address 127.4.48.254:46733)
0811 12:10:29.650859 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:29.650865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:29.652417 (+ 1552us) server_negotiation.cc:407] Connection header received
0811 12:10:29.654100 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:29.654105 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:29.654179 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:29.654266 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:29.655513 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:29.655519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:29.655524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:29.655543 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:29.655579 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:29.655590 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:29.655716 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:29.655894 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 12:10:29.655968 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:10:29.657094 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:29.651167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46733 (local address 127.4.48.194:43039)
0811 12:10:29.652252 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:29.652298 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:29.653853 (+ 1555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:29.654500 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:29.654507 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:29.654591 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:29.655225 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:29.655227 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:29.655262 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:29.655353 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:29.655697 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:29.655709 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:29.655716 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:10:29.655967 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 12:10:29.656200 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":346,"threads_started":1}
I20260811 12:10:29.658339 21888 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46733
I20260811 12:10:29.658612 21888 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:29.659202 21888 heartbeater.cc:511] Master 127.4.48.254:46733 requested a full tablet report, sending...
I20260811 12:10:29.660792 21303 ts_manager.cc:194] Registered new tserver with Master: cd7ae48eb74e4ea5a36e8bd19c8046cf (127.4.48.194:38845)
W20260811 12:10:29.985075 21893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:29.985314 21893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:29.985435 21893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:29.992363 21893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:29.992448 21893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:29.992584 21893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:30.001266 21893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:46733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:30.002516 21893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:30.004235 21893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:30.016546 21907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:30.016631 21915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:30.016567 21910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:30.017381 21893 server_base.cc:1034] running on GCE node
I20260811 12:10:30.017933 21893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:30.018859 21893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:30.020061 21893 hybrid_clock.cc:648] HybridClock initialized: now 1786450230020000 us; error 41 us; skew 500 ppm
I20260811 12:10:30.020395 21893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:10:30.022594 21893 webserver.cc:460] Webserver started at http://127.4.48.195:46531/ using document root <none> and password file <none>
I20260811 12:10:30.023267 21893 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:30.023372 21893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:30.023698 21893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:30.026371 21893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "c89e9337766a43a18b1caa716e82c409"
format_stamp: "Formatted at 2026-08-11 12:10:30 on dist-test-slave-nj21"
server_key: "c934234eddb97190ccdfc204411ffc91"
server_key_iv: "f33e82accb7f9e4c8363212977e5d402"
server_key_version: "encryptionkey@0"
I20260811 12:10:30.027110 21893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "c89e9337766a43a18b1caa716e82c409"
format_stamp: "Formatted at 2026-08-11 12:10:30 on dist-test-slave-nj21"
server_key: "c934234eddb97190ccdfc204411ffc91"
server_key_iv: "f33e82accb7f9e4c8363212977e5d402"
server_key_version: "encryptionkey@0"
I20260811 12:10:30.032076 21893 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 12:10:30.035328 21925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:30.036616 21893 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:10:30.036768 21893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "c89e9337766a43a18b1caa716e82c409"
format_stamp: "Formatted at 2026-08-11 12:10:30 on dist-test-slave-nj21"
server_key: "c934234eddb97190ccdfc204411ffc91"
server_key_iv: "f33e82accb7f9e4c8363212977e5d402"
server_key_version: "encryptionkey@0"
I20260811 12:10:30.036984 21893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:30.042255 21893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:30.042991 21893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:30.043221 21893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:30.044104 21893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:30.045415 21893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:30.045509 21893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:30.045617 21893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:30.045686 21893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:30.074087 21893 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:41015
I20260811 12:10:30.074137 22112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:41015 every 8 connection(s)
I20260811 12:10:30.075333 21893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:30.079437 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 21893
I20260811 12:10:30.079633 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:10:30.080196 4291 external_mini_cluster.cc:1428] Setting key e31e0964f7935bbae6f5e82e6b35d6bb
I20260811 12:10:30.083734 21639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:30.077770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:60943 (local address 127.4.48.254:46733)
0811 12:10:30.077973 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:30.077977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:30.079364 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 12:10:30.081465 (+ 2101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:30.081470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:30.081530 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:30.081619 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:30.082997 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:30.083002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:30.083005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:30.083022 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:30.083053 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:30.083064 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:30.083172 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:30.083361 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 12:10:30.083428 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:10:30.084605 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:30.078205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46733 (local address 127.4.48.195:60943)
0811 12:10:30.079165 (+ 960us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:30.079218 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:30.081155 (+ 1937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:30.081874 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:30.081882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:30.081969 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:30.082687 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:10:30.082690 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:30.082727 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:10:30.082812 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:30.083154 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:30.083164 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:30.083172 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:10:30.083433 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 12:10:30.083693 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":279,"threads_started":1}
I20260811 12:10:30.085798 22113 heartbeater.cc:347] Connected to a master server at 127.4.48.254:46733
I20260811 12:10:30.086048 22113 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:30.086567 22113 heartbeater.cc:511] Master 127.4.48.254:46733 requested a full tablet report, sending...
I20260811 12:10:30.087893 21303 ts_manager.cc:194] Registered new tserver with Master: c89e9337766a43a18b1caa716e82c409 (127.4.48.195:41015)
I20260811 12:10:30.095281 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 12:10:30.101329 21639 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:10:30.099787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43670 (local address 127.4.48.254:46733)
0811 12:10:30.099946 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:30.099951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:30.100459 (+ 508us) server_negotiation.cc:407] Connection header received
0811 12:10:30.100628 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:30.100632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:30.100686 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:30.100774 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:30.101130 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43670: BlockingRecv error: recv got EOF from 127.0.0.1:43670 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:10:30.102087 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 21413
I20260811 12:10:30.105432 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:30.103044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49088 (local address 127.4.48.193:40541)
0811 12:10:30.103543 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:30.103550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:30.103575 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:10:30.103638 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:30.103644 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:30.103890 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:30.104013 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:30.104829 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:30.104836 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:30.104847 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:30.104865 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:30.104904 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:30.104917 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:30.105019 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:30.105069 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 12:10:30.105146 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":220,"threads_started":1}
I20260811 12:10:30.106613 21622 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:30.190500 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 21413
I20260811 12:10:30.214123 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 21641
I20260811 12:10:30.217429 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:30.214994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56398 (local address 127.4.48.194:38845)
0811 12:10:30.215493 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:30.215500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:30.215528 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:10:30.215598 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:30.215604 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:30.215876 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:30.216007 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:30.216807 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:30.216813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:30.216834 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:30.216846 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:30.216888 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:30.216905 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:30.217009 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:30.217056 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 12:10:30.217153 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":224,"threads_started":1}
I20260811 12:10:30.218688 21874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:30.297873 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 21641
I20260811 12:10:30.319600 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 21893
I20260811 12:10:30.323453 22136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:30.320732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47142 (local address 127.4.48.195:41015)
0811 12:10:30.321396 (+ 664us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:30.321403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:30.321433 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:10:30.321494 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:30.321501 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:30.321711 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:30.321845 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:30.322746 (+ 901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:30.322752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:30.322764 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:10:30.322780 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:30.322818 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:10:30.322848 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:30.322991 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:30.323049 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 12:10:30.323138 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":338,"threads_started":1}
I20260811 12:10:30.324753 22107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:30.404525 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 21893
I20260811 12:10:30.426810 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 21258
I20260811 12:10:30.428082 21364 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:30.528174 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 21258
2026-08-11T12:10:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2337 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:10:30 dist-test-slave-nj21 krb5kdc[22143](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:10:30 dist-test-slave-nj21 krb5kdc[22143](info): set up 2 sockets
Aug 11 12:10:30 dist-test-slave-nj21 krb5kdc[22143](info): commencing operation
krb5kdc: starting...
W20260811 12:10:31.757678 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.181s user 0.006s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:10:31 dist-test-slave-nj21 krb5kdc[22143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:10:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:10:32.011193 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44871
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:44871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:32.334859 22163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:32.335129 22163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:32.335265 22163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:32.342234 22163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:10:32.342325 22163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:32.342394 22163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:32.342476 22163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:10:32.342535 22163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:10:32.352021 22163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:44871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:32.353267 22163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:32.354521 22163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:32.365638 22177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:32.365651 22178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:32.365689 22180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:32.366480 22163 server_base.cc:1034] running on GCE node
I20260811 12:10:32.367185 22163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:32.368548 22163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:32.369772 22163 hybrid_clock.cc:648] HybridClock initialized: now 1786450232369713 us; error 35 us; skew 500 ppm
I20260811 12:10:32.370106 22163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:32 dist-test-slave-nj21 krb5kdc[22143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450232, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:32.378577 22163 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:10:32.380089 22163 webserver.cc:460] Webserver started at http://127.4.48.254:33607/ using document root <none> and password file <none>
I20260811 12:10:32.380771 22163 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:32.380894 22163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:32.381179 22163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:32.384141 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1"
format_stamp: "Formatted at 2026-08-11 12:10:32 on dist-test-slave-nj21"
server_key: "9b7301411f05c8901e2d0ca8ad267923"
server_key_iv: "8bc49594229e493c491755266d0ff5a3"
server_key_version: "encryptionkey@0"
I20260811 12:10:32.384902 22163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1"
format_stamp: "Formatted at 2026-08-11 12:10:32 on dist-test-slave-nj21"
server_key: "9b7301411f05c8901e2d0ca8ad267923"
server_key_iv: "8bc49594229e493c491755266d0ff5a3"
server_key_version: "encryptionkey@0"
I20260811 12:10:32.389838 22163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:10:32.393148 22190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:32.394317 22163 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:10:32.394474 22163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1"
format_stamp: "Formatted at 2026-08-11 12:10:32 on dist-test-slave-nj21"
server_key: "9b7301411f05c8901e2d0ca8ad267923"
server_key_iv: "8bc49594229e493c491755266d0ff5a3"
server_key_version: "encryptionkey@0"
I20260811 12:10:32.394646 22163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:32.406625 22163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:32.410458 22163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:32.410694 22163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:32.427525 22163 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:44871
I20260811 12:10:32.427558 22272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:44871 every 8 connection(s)
I20260811 12:10:32.428777 22163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:10:32.431934 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 22163
I20260811 12:10:32.432171 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:10:32.432556 22273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:32.432988 4291 external_mini_cluster.cc:1428] Setting key b1592b6b352fe2ba34072682870c5309
I20260811 12:10:32.446684 22273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: Bootstrap starting.
Aug 11 12:10:32 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:32.450340 22273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:32.451659 22273 log.cc:824] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:32.455225 22273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: No bootstrap required, opened a new log
I20260811 12:10:32.456813 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:32.434888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35054 (local address 127.4.48.254:44871)
0811 12:10:32.435730 (+ 842us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:32.435745 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:32.435786 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:10:32.437266 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:32.437286 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:32.437621 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:32.437918 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:32.439757 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:32.441005 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:32.442788 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:32.443473 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:32.443601 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:32.448214 (+ 4613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:32.448225 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:32.448242 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:32.448299 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:32.452064 (+ 3765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:32.453013 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:32.453019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:32.453025 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:32.453223 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:32.453615 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:32.453619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:32.453622 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:32.454313 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:32.454637 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:32.455039 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 12:10:32.455492 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":368,"threads_started":1}
I20260811 12:10:32.463438 22273 raft_consensus.cc:348] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } }
I20260811 12:10:32.463786 22273 raft_consensus.cc:374] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:32.463877 22273 raft_consensus.cc:729] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b835a95bebf74bbe9eb2a3cf14cda3d1, State: Initialized, Role: FOLLOWER
I20260811 12:10:32.464430 22273 consensus_queue.cc:260] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } }
I20260811 12:10:32.464697 22273 raft_consensus.cc:388] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:10:32.464838 22273 raft_consensus.cc:482] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:10:32.465020 22273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:32.467052 22273 raft_consensus.cc:504] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } }
I20260811 12:10:32.467520 22273 leader_election.cc:302] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b835a95bebf74bbe9eb2a3cf14cda3d1; no voters:
I20260811 12:10:32.468225 22273 leader_election.cc:288] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:10:32.468350 22281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:32.468978 22281 raft_consensus.cc:686] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [term 1 LEADER]: Becoming Leader. State: Replica: b835a95bebf74bbe9eb2a3cf14cda3d1, State: Running, Role: LEADER
I20260811 12:10:32.469483 22281 consensus_queue.cc:237] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } }
I20260811 12:10:32.470140 22273 sys_catalog.cc:564] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:10:32.472119 22283 sys_catalog.cc:455] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b835a95bebf74bbe9eb2a3cf14cda3d1. Latest consensus state: current_term: 1 leader_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } } }
I20260811 12:10:32.472515 22283 sys_catalog.cc:458] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:32.472436 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b835a95bebf74bbe9eb2a3cf14cda3d1" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44871 } } }
I20260811 12:10:32.472781 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1 [sys.catalog]: This master's current role is: LEADER
I20260811 12:10:32.473414 22291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:10:32.478322 22291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:10:32.486873 22291 catalog_manager.cc:1347] Generated new cluster ID: 7fd790bc6b1543bb8bc9f4b672b7a9e3
I20260811 12:10:32.486980 22291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:10:32.495321 22291 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:10:32.496665 22291 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:10:32.510370 22291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: Generated new TSK 0
I20260811 12:10:32.511092 22291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:10:32.583117 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:32.967573 22311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:32.967882 22311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:32.967990 22311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:32.977496 22311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:32.977628 22311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:32.977802 22311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:10:32.991641 22311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:32.993440 22311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:32.995340 22311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:33.012024 22320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:33.012676 22311 server_base.cc:1034] running on GCE node
W20260811 12:10:33.012729 22323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:33.013247 22321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:33.013880 22311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:33.014724 22311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:33.015918 22311 hybrid_clock.cc:648] HybridClock initialized: now 1786450233015868 us; error 35 us; skew 500 ppm
I20260811 12:10:33.016187 22311 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:33 dist-test-slave-nj21 krb5kdc[22143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450233, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:33.019075 22311 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:10:33.020394 22311 webserver.cc:460] Webserver started at http://127.4.48.193:38581/ using document root <none> and password file <none>
I20260811 12:10:33.021093 22311 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:33.021193 22311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:33.021445 22311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:33.024211 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "a8d61528cff941349a0078a3862255f3"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "a3ceeb1c81f97c4d1ab8e5b1ee89dc38"
server_key_iv: "fc61527cf0576aafad2c5e6420ed646e"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.025003 22311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "a8d61528cff941349a0078a3862255f3"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "a3ceeb1c81f97c4d1ab8e5b1ee89dc38"
server_key_iv: "fc61527cf0576aafad2c5e6420ed646e"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.029888 22311 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:10:33.033141 22343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.034377 22311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:10:33.034507 22311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "a8d61528cff941349a0078a3862255f3"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "a3ceeb1c81f97c4d1ab8e5b1ee89dc38"
server_key_iv: "fc61527cf0576aafad2c5e6420ed646e"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.034655 22311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:33.053354 22311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:33.056926 22311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:33.057150 22311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:33.057971 22311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:33.059190 22311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:33.059260 22311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.059357 22311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:33.059401 22311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.088871 22311 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35417
I20260811 12:10:33.088909 22514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35417 every 8 connection(s)
I20260811 12:10:33.090116 22311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:10:33.093887 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 22311
I20260811 12:10:33.094127 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:10:33.094938 4291 external_mini_cluster.cc:1428] Setting key 89e4c136abd356673092cf9bc4a3f612
Aug 11 12:10:33 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450233, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:33.111492 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:33.092547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:42229 (local address 127.4.48.254:44871)
0811 12:10:33.092982 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:33.092986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:33.093961 (+ 975us) server_negotiation.cc:407] Connection header received
0811 12:10:33.096308 (+ 2347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:33.096314 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:33.096404 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:33.096506 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:33.098451 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.099306 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.101343 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.101741 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.101864 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:33.105753 (+ 3889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:33.105758 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:33.105761 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:33.105796 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:33.108386 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:33.109184 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:33.109190 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:33.109194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:33.109343 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:33.109743 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:33.109748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:33.109751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:33.110095 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:33.110246 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:33.110900 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 12:10:33.111123 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":226,"threads_started":1}
I20260811 12:10:33.112238 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:33.092977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44871 (local address 127.4.48.193:42229)
0811 12:10:33.093750 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:33.093800 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:33.096044 (+ 2244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:33.096783 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:33.096794 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:33.097487 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:33.098286 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:33.098309 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.099469 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.099475 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:33.101210 (+ 1735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:33.101224 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.101990 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.101996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:33.102250 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:33.102934 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:33.102968 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:33.105514 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:33.108521 (+ 3007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:33.108530 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:33.108545 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:33.109061 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:33.109463 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:33.109467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:33.109469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:33.109639 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:33.110201 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:33.110207 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:33.110515 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 12:10:33.110741 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 12:10:33.111133 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":273,"threads_started":1}
I20260811 12:10:33.113689 22517 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44871
I20260811 12:10:33.114022 22517 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:33.114902 22517 heartbeater.cc:511] Master 127.4.48.254:44871 requested a full tablet report, sending...
I20260811 12:10:33.117249 22215 ts_manager.cc:194] Registered new tserver with Master: a8d61528cff941349a0078a3862255f3 (127.4.48.193:35417)
I20260811 12:10:33.119349 22215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:42229
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:10:33.157042 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:33.480837 22528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:33.481153 22528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:33.481293 22528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:33.488214 22528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:33.488332 22528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:33.488477 22528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:10:33.497784 22528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:33.499034 22528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:33.500308 22528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:33.512459 22539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:33.512542 22534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:33.512509 22536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:33.513501 22528 server_base.cc:1034] running on GCE node
I20260811 12:10:33.514091 22528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:33.514925 22528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:33.516119 22528 hybrid_clock.cc:648] HybridClock initialized: now 1786450233516068 us; error 41 us; skew 500 ppm
I20260811 12:10:33.516410 22528 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:33 dist-test-slave-nj21 krb5kdc[22143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450233, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:33.519374 22528 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:10:33.520674 22528 webserver.cc:460] Webserver started at http://127.4.48.194:40477/ using document root <none> and password file <none>
I20260811 12:10:33.521296 22528 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:33.521414 22528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:33.521687 22528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:33.524534 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "5fb200d349d64e22977546ea2a79581d"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "3407a0ae965f65c7c89a988cf77a3f04"
server_key_iv: "86aca8b08a46ced69ce5d232b44330f7"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.525274 22528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "5fb200d349d64e22977546ea2a79581d"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "3407a0ae965f65c7c89a988cf77a3f04"
server_key_iv: "86aca8b08a46ced69ce5d232b44330f7"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.530126 22528 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:10:33.533574 22553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.534788 22528 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:10:33.534952 22528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "5fb200d349d64e22977546ea2a79581d"
format_stamp: "Formatted at 2026-08-11 12:10:33 on dist-test-slave-nj21"
server_key: "3407a0ae965f65c7c89a988cf77a3f04"
server_key_iv: "86aca8b08a46ced69ce5d232b44330f7"
server_key_version: "encryptionkey@0"
I20260811 12:10:33.535125 22528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:33.577427 22528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:33.581544 22528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:33.581826 22528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:33.582736 22528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:33.584086 22528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:33.584167 22528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.584287 22528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:33.584342 22528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:33.613862 22528 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:40373
I20260811 12:10:33.613916 22744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:40373 every 8 connection(s)
I20260811 12:10:33.615056 22528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:10:33.619912 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 22528
I20260811 12:10:33.620100 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:10:33.620518 4291 external_mini_cluster.cc:1428] Setting key 1e2d8a84bc754fede2b0b2a6dd50152e
Aug 11 12:10:33 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450233, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:33.634390 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:33.617447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:37515 (local address 127.4.48.254:44871)
0811 12:10:33.617869 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:33.617873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:33.618787 (+ 914us) server_negotiation.cc:407] Connection header received
0811 12:10:33.620674 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:33.620681 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:33.620769 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:33.620905 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:33.622461 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.623501 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.625008 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.625574 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.625694 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:33.628830 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:33.628835 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:33.628839 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:33.628876 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:33.631720 (+ 2844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:33.632354 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:33.632359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:33.632363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:33.632517 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:33.632880 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:33.632884 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:33.632887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:33.633202 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:33.633333 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:33.633923 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 12:10:33.634122 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":209,"threads_started":1}
I20260811 12:10:33.635496 22750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:33.617972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44871 (local address 127.4.48.194:37515)
0811 12:10:33.618641 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:33.618678 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:33.620410 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:33.621109 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:33.621120 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:33.621686 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:33.622267 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:33.622292 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.623630 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.623634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:33.624867 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:33.624880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:33.625751 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:33.625756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:33.625937 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:33.626529 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:33.626551 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:33.628613 (+ 2062us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:33.631833 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:33.631839 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:33.631849 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:33.632236 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:33.632605 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:33.632608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:33.632610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:33.632774 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:33.633307 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:33.633312 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:33.633590 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 12:10:33.633788 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 12:10:33.634187 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":267,"threads_started":1}
I20260811 12:10:33.636842 22746 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44871
I20260811 12:10:33.637125 22746 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:33.637848 22746 heartbeater.cc:511] Master 127.4.48.254:44871 requested a full tablet report, sending...
I20260811 12:10:33.639370 22215 ts_manager.cc:194] Registered new tserver with Master: 5fb200d349d64e22977546ea2a79581d (127.4.48.194:40373)
I20260811 12:10:33.640218 22215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:37515
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:10:33.678936 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:10:34.001606 22755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:10:34.001866 22755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:10:34.001958 22755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:10:34.008836 22755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:10:34.008929 22755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:10:34.009040 22755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:10:34.018013 22755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44871
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:10:34.019091 22755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:10:34.020303 22755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:10:34.032588 22761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:34.032650 22765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:10:34.032675 22762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:10:34.033699 22755 server_base.cc:1034] running on GCE node
I20260811 12:10:34.034240 22755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:10:34.035100 22755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:10:34.036296 22755 hybrid_clock.cc:648] HybridClock initialized: now 1786450234036254 us; error 29 us; skew 500 ppm
I20260811 12:10:34.036589 22755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:10:34 dist-test-slave-nj21 krb5kdc[22143](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450234, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:10:34.039661 22755 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:10:34.041000 22755 webserver.cc:460] Webserver started at http://127.4.48.195:37905/ using document root <none> and password file <none>
I20260811 12:10:34.041632 22755 fs_manager.cc:359] Metadata directory not provided
I20260811 12:10:34.041747 22755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:10:34.042028 22755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:10:34.044946 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "612f4958e507410ab58038108a32f425"
format_stamp: "Formatted at 2026-08-11 12:10:34 on dist-test-slave-nj21"
server_key: "93c0c54aa966fa6b1ad46fa33d3dfc49"
server_key_iv: "25fc6ac37c103acaec331f10bd00de3c"
server_key_version: "encryptionkey@0"
I20260811 12:10:34.045691 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "612f4958e507410ab58038108a32f425"
format_stamp: "Formatted at 2026-08-11 12:10:34 on dist-test-slave-nj21"
server_key: "93c0c54aa966fa6b1ad46fa33d3dfc49"
server_key_iv: "25fc6ac37c103acaec331f10bd00de3c"
server_key_version: "encryptionkey@0"
I20260811 12:10:34.050525 22755 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:10:34.053696 22773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:34.054900 22755 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:10:34.055042 22755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "612f4958e507410ab58038108a32f425"
format_stamp: "Formatted at 2026-08-11 12:10:34 on dist-test-slave-nj21"
server_key: "93c0c54aa966fa6b1ad46fa33d3dfc49"
server_key_iv: "25fc6ac37c103acaec331f10bd00de3c"
server_key_version: "encryptionkey@0"
I20260811 12:10:34.055208 22755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:10:34.068221 22755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:10:34.071852 22755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:10:34.072106 22755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:10:34.072992 22755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:10:34.074268 22755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:10:34.074354 22755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:34.074473 22755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:10:34.074532 22755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:10:34.104949 22755 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:41515
I20260811 12:10:34.104995 22952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:41515 every 8 connection(s)
I20260811 12:10:34.106215 22755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:10:34.113121 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 22755
I20260811 12:10:34.113325 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:10:34.113751 4291 external_mini_cluster.cc:1428] Setting key b9eaef60834cd04130fe45891717d663
Aug 11 12:10:34 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450234, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:10:34.122737 22517 heartbeater.cc:503] Master 127.4.48.254:44871 was elected leader, sending a full tablet report...
I20260811 12:10:34.125743 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.108622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:40419 (local address 127.4.48.254:44871)
0811 12:10:34.108846 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.108852 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.109968 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 12:10:34.111623 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.111628 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.111733 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.111814 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:34.113365 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.114137 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.115975 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.116366 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.116487 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.119664 (+ 3177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:34.119688 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:34.119692 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:34.119731 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:34.122921 (+ 3190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.123758 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.123764 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.123768 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.123890 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.124369 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.124374 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.124376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.124671 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:34.124787 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.125355 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.125518 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:10:34.126716 22956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.109113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44871 (local address 127.4.48.195:40419)
0811 12:10:34.109817 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.109860 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.111396 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.112014 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.112021 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.112500 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:10:34.113195 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.113223 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.114283 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.114286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.115849 (+ 1563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.115865 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.116589 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.116598 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.116837 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.117439 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:10:34.117461 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:10:34.119485 (+ 2024us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:10:34.123046 (+ 3561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:34.123055 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:34.123070 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:34.123618 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:34.123997 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:10:34.124001 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:10:34.124004 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:10:34.124248 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:10:34.124767 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:10:34.124773 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:10:34.125065 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.125274 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.125635 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":281,"threads_started":1}
I20260811 12:10:34.127983 22954 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44871
I20260811 12:10:34.128258 22954 heartbeater.cc:464] Registering TS with master...
I20260811 12:10:34.128954 22954 heartbeater.cc:511] Master 127.4.48.254:44871 requested a full tablet report, sending...
I20260811 12:10:34.130239 22215 ts_manager.cc:194] Registered new tserver with Master: 612f4958e507410ab58038108a32f425 (127.4.48.195:41515)
I20260811 12:10:34.130973 22215 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:40419
I20260811 12:10:34.140784 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:10:34.156615 22749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.145029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.4.48.254:44871)
0811 12:10:34.145179 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.145183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.145764 (+ 581us) server_negotiation.cc:407] Connection header received
0811 12:10:34.145903 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.145907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.145971 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.146049 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:34.147384 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.148166 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.149301 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.149721 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.149810 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.151907 (+ 2097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:34.151911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:34.151915 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:34.151945 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:34.154377 (+ 2432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.154943 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.154947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.154949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.155091 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.155367 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.155370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.155372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.155663 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:34.155890 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.156186 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.156350 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:10:34.160501 22215 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:10:34.167711 22215 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:10:34.203866 22974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.181269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35417 (local address 127.0.0.1:53784)
0811 12:10:34.195872 (+ 14603us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.195891 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.196024 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.196683 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.196691 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.196719 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:34.197132 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.197145 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.199088 (+ 1943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.199093 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.201304 (+ 2211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.201318 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.202834 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.202841 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.203141 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.203201 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.203313 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.203443 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14480,"mutex_wait_us":47,"thread_start_us":14275,"threads_started":1}
I20260811 12:10:34.204495 22969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.179713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:40373 (local address 127.0.0.1:37296)
0811 12:10:34.193529 (+ 13816us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.193549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.193679 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.197392 (+ 3713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.197397 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.197435 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:34.197900 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.197916 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.199907 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.199911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.202053 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.202072 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.203732 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.203739 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.203915 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.203946 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.204044 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.204165 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13681,"thread_start_us":12198,"threads_started":1}
I20260811 12:10:34.204777 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.181925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53784 (local address 127.4.48.193:35417)
0811 12:10:34.195296 (+ 13371us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.195307 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.196013 (+ 706us) server_negotiation.cc:407] Connection header received
0811 12:10:34.196171 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.196180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.196469 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.196657 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:34.197341 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.198915 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.201496 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.202623 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.202755 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.202995 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.204302 (+ 1307us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.204470 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13164,"thread_start_us":12999,"threads_started":1}
I20260811 12:10:34.205360 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.179794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37296 (local address 127.4.48.194:40373)
0811 12:10:34.196549 (+ 16755us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.196558 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.196588 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:10:34.196689 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.196697 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.196920 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.197113 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:34.198101 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.199776 (+ 1675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.202225 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.203495 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.203627 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.203841 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.204864 (+ 1023us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.205037 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16582,"thread_start_us":16441,"threads_started":1}
I20260811 12:10:34.208289 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.179666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.4.48.195:41515)
0811 12:10:34.198530 (+ 18864us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.198535 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.198560 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:10:34.198631 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.198635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.198776 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.198940 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:34.199582 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.201155 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.203337 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.204534 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.204652 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.204887 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.207815 (+ 2928us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.207979 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18690,"thread_start_us":18523,"threads_started":1}
I20260811 12:10:34.208622 22970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.179372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41515 (local address 127.0.0.1:49224)
0811 12:10:34.192305 (+ 12933us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.192352 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.192524 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.198979 (+ 6455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.198983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.199008 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:34.199414 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.199424 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.201336 (+ 1912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.201341 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.203175 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.203189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.205019 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.205025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.205212 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.205246 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.205329 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.205446 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12682,"mutex_wait_us":65,"thread_start_us":14390,"threads_started":1}
I20260811 12:10:34.208320 22420 tablet_service.cc:1467] Processing CreateTablet for tablet 16372ed3a96844e2a787612fdeff1146 (DEFAULT_TABLE table=test-table [id=e914a0a6c6c0476783c26829d903863c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:34.208387 22637 tablet_service.cc:1467] Processing CreateTablet for tablet 16372ed3a96844e2a787612fdeff1146 (DEFAULT_TABLE table=test-table [id=e914a0a6c6c0476783c26829d903863c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:34.209702 22420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16372ed3a96844e2a787612fdeff1146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:34.209811 22637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16372ed3a96844e2a787612fdeff1146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:34.212239 22853 tablet_service.cc:1467] Processing CreateTablet for tablet 16372ed3a96844e2a787612fdeff1146 (DEFAULT_TABLE table=test-table [id=e914a0a6c6c0476783c26829d903863c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:10:34.213528 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16372ed3a96844e2a787612fdeff1146. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:10:34.220708 22977 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Bootstrap starting.
I20260811 12:10:34.224061 22977 tablet_bootstrap.cc:654] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:34.225019 22977 log.cc:824] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:34.228456 22979 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Bootstrap starting.
I20260811 12:10:34.230150 22983 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Bootstrap starting.
I20260811 12:10:34.232487 22979 tablet_bootstrap.cc:654] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:34.233146 22983 tablet_bootstrap.cc:654] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Neither blocks nor log segments found. Creating new log.
I20260811 12:10:34.233814 22979 log.cc:824] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:34.234359 22983 log.cc:824] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Log is configured to *not* fsync() on all Append() calls
I20260811 12:10:34.236657 22977 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: No bootstrap required, opened a new log
I20260811 12:10:34.236917 22977 ts_tablet_manager.cc:1397] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Time spent bootstrapping tablet: real 0.016s user 0.015s sys 0.000s
I20260811 12:10:34.237293 22983 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: No bootstrap required, opened a new log
I20260811 12:10:34.237344 22979 tablet_bootstrap.cc:492] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: No bootstrap required, opened a new log
I20260811 12:10:34.237576 22983 ts_tablet_manager.cc:1397] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:10:34.237593 22979 ts_tablet_manager.cc:1397] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:10:34.244593 22977 raft_consensus.cc:348] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.244938 22977 raft_consensus.cc:374] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:34.245034 22977 raft_consensus.cc:729] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb200d349d64e22977546ea2a79581d, State: Initialized, Role: FOLLOWER
I20260811 12:10:34.245582 22977 consensus_queue.cc:260] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.246582 22746 heartbeater.cc:503] Master 127.4.48.254:44871 was elected leader, sending a full tablet report...
I20260811 12:10:34.247843 22979 raft_consensus.cc:348] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.248258 22979 raft_consensus.cc:374] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:34.248354 22979 raft_consensus.cc:729] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8d61528cff941349a0078a3862255f3, State: Initialized, Role: FOLLOWER
I20260811 12:10:34.248440 22977 ts_tablet_manager.cc:1428] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 12:10:34.248268 22983 raft_consensus.cc:348] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.248785 22983 raft_consensus.cc:374] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:10:34.248886 22983 raft_consensus.cc:729] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 612f4958e507410ab58038108a32f425, State: Initialized, Role: FOLLOWER
I20260811 12:10:34.248930 22979 consensus_queue.cc:260] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.249473 22983 consensus_queue.cc:260] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.250617 22979 ts_tablet_manager.cc:1428] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260811 12:10:34.252354 22954 heartbeater.cc:503] Master 127.4.48.254:44871 was elected leader, sending a full tablet report...
I20260811 12:10:34.253849 22983 ts_tablet_manager.cc:1428] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Time spent starting tablet: real 0.016s user 0.012s sys 0.003s
W20260811 12:10:34.342475 22518 tablet.cc:2369] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:10:34.357705 22955 tablet.cc:2369] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:10:34.366947 22748 tablet.cc:2369] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:10:34.377157 22987 raft_consensus.cc:482] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:10:34.377478 22987 raft_consensus.cc:504] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.379124 22987 leader_election.cc:288] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 612f4958e507410ab58038108a32f425 (127.4.48.195:41515), a8d61528cff941349a0078a3862255f3 (127.4.48.193:35417)
I20260811 12:10:34.385726 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.379485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:43275 (local address 127.4.48.195:41515)
0811 12:10:34.379714 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.379719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.380568 (+ 849us) server_negotiation.cc:407] Connection header received
0811 12:10:34.380853 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.380857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.380932 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.381012 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:34.381564 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.382318 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.384120 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.384885 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.385037 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.385111 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.385364 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.385489 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:10:34.386053 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.379722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:35401 (local address 127.4.48.193:35417)
0811 12:10:34.379952 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.379958 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.380913 (+ 955us) server_negotiation.cc:407] Connection header received
0811 12:10:34.381124 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.381130 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.381207 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.381267 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:10:34.381959 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.382715 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.384440 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.385207 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.385360 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.385434 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.385704 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.385829 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:10:34.386077 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.379554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41515 (local address 127.4.48.194:43275)
0811 12:10:34.380443 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.380463 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.380699 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.381056 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.381060 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.381086 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:34.381420 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.381434 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.382448 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.382454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.383973 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.383986 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.385091 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.385097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.385222 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.385246 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.385335 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.385464 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":725,"mutex_wait_us":2,"thread_start_us":513,"threads_started":1}
I20260811 12:10:34.386204 22993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.379586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35417 (local address 127.4.48.194:35401)
0811 12:10:34.380796 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0811 12:10:34.380814 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:10:34.380960 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:10:34.381471 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:10:34.381474 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:10:34.381498 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:10:34.381801 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.381809 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.382860 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.382863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.384274 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:10:34.384287 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.385421 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.385427 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:10:34.385562 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.385588 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 12:10:34.385665 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 12:10:34.385785 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1089,"mutex_wait_us":1,"spinlock_wait_cycles":23808,"thread_start_us":830,"threads_started":1}
I20260811 12:10:34.387075 22878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16372ed3a96844e2a787612fdeff1146" candidate_uuid: "5fb200d349d64e22977546ea2a79581d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "612f4958e507410ab58038108a32f425" is_pre_election: true
I20260811 12:10:34.387081 22447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16372ed3a96844e2a787612fdeff1146" candidate_uuid: "5fb200d349d64e22977546ea2a79581d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d61528cff941349a0078a3862255f3" is_pre_election: true
I20260811 12:10:34.387537 22878 raft_consensus.cc:2454] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb200d349d64e22977546ea2a79581d in term 0.
I20260811 12:10:34.387538 22447 raft_consensus.cc:2454] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fb200d349d64e22977546ea2a79581d in term 0.
I20260811 12:10:34.388181 22554 leader_election.cc:302] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fb200d349d64e22977546ea2a79581d, a8d61528cff941349a0078a3862255f3; no voters:
I20260811 12:10:34.388527 22987 raft_consensus.cc:2781] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:10:34.388680 22987 raft_consensus.cc:482] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:10:34.388782 22987 raft_consensus.cc:3037] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:34.390908 22987 raft_consensus.cc:504] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.391418 22987 leader_election.cc:288] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [CANDIDATE]: Term 1 election: Requested vote from peers 612f4958e507410ab58038108a32f425 (127.4.48.195:41515), a8d61528cff941349a0078a3862255f3 (127.4.48.193:35417)
I20260811 12:10:34.391861 22878 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16372ed3a96844e2a787612fdeff1146" candidate_uuid: "5fb200d349d64e22977546ea2a79581d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "612f4958e507410ab58038108a32f425"
I20260811 12:10:34.392056 22878 raft_consensus.cc:3037] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:34.392019 22447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16372ed3a96844e2a787612fdeff1146" candidate_uuid: "5fb200d349d64e22977546ea2a79581d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a8d61528cff941349a0078a3862255f3"
I20260811 12:10:34.392218 22447 raft_consensus.cc:3037] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:10:34.395157 22878 raft_consensus.cc:2454] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb200d349d64e22977546ea2a79581d in term 1.
I20260811 12:10:34.395176 22447 raft_consensus.cc:2454] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fb200d349d64e22977546ea2a79581d in term 1.
I20260811 12:10:34.395596 22557 leader_election.cc:302] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fb200d349d64e22977546ea2a79581d, 612f4958e507410ab58038108a32f425; no voters:
I20260811 12:10:34.400548 22987 raft_consensus.cc:2781] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:10:34.402535 22987 raft_consensus.cc:686] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 1 LEADER]: Becoming Leader. State: Replica: 5fb200d349d64e22977546ea2a79581d, State: Running, Role: LEADER
I20260811 12:10:34.403097 22987 consensus_queue.cc:237] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } }
I20260811 12:10:34.407470 22215 catalog_manager.cc:5495] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fb200d349d64e22977546ea2a79581d (127.4.48.194). New cstate: current_term: 1 leader_uuid: "5fb200d349d64e22977546ea2a79581d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5fb200d349d64e22977546ea2a79581d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 40373 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 } health_report { overall_health: UNKNOWN } } }
I20260811 12:10:34.439042 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.433960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37308 (local address 127.4.48.194:40373)
0811 12:10:34.434160 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.434165 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.434184 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:10:34.434305 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.434308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.434402 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.434523 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:10:34.434890 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.435615 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.436940 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.437335 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.437448 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.437922 (+ 474us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:10:34.438441 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:10:34.438570 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.438665 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.438746 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:10:34.447520 22447 raft_consensus.cc:1261] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 1 FOLLOWER]: Refusing update from remote peer 5fb200d349d64e22977546ea2a79581d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:34.447546 22878 raft_consensus.cc:1261] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 1 FOLLOWER]: Refusing update from remote peer 5fb200d349d64e22977546ea2a79581d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:10:34.448447 22994 consensus_queue.cc:1035] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a8d61528cff941349a0078a3862255f3" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:34.448762 22987 consensus_queue.cc:1035] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [LEADER]: Connected to new peer: Peer: permanent_uuid: "612f4958e507410ab58038108a32f425" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:10:34.458806 23001 mvcc.cc:205] Tried to move back new op lower bound from 7317300160288075776 to 7317300160119373824. Current Snapshot: MvccSnapshot[applied={T|T < 7317300160288075776}]
I20260811 12:10:34.465698 23003 mvcc.cc:205] Tried to move back new op lower bound from 7317300160288075776 to 7317300160119373824. Current Snapshot: MvccSnapshot[applied={T|T < 7317300160288075776}]
I20260811 12:10:34.473273 22215 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:34.473503 22215 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35060:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:10:34.476794 22215 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b835a95bebf74bbe9eb2a3cf14cda3d1: Sending DeleteTablet for 3 replicas of tablet 16372ed3a96844e2a787612fdeff1146
I20260811 12:10:34.477845 22853 tablet_service.cc:1514] Processing DeleteTablet for tablet 16372ed3a96844e2a787612fdeff1146 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:34 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:49224
I20260811 12:10:34.477972 22420 tablet_service.cc:1514] Processing DeleteTablet for tablet 16372ed3a96844e2a787612fdeff1146 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:34 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:53784
I20260811 12:10:34.478111 22637 tablet_service.cc:1514] Processing DeleteTablet for tablet 16372ed3a96844e2a787612fdeff1146 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:10:34 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:37296
I20260811 12:10:34.480365 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 22311
I20260811 12:10:34.488116 23020 tablet_replica.cc:330] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: stopping tablet replica
I20260811 12:10:34.488655 23020 raft_consensus.cc:2229] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:10:34.489418 23020 raft_consensus.cc:2258] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:34.490060 23018 tablet_replica.cc:330] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: stopping tablet replica
I20260811 12:10:34.490724 23018 raft_consensus.cc:2229] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:34.491283 23018 raft_consensus.cc:2258] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:10:34.494026 23020 ts_tablet_manager.cc:1905] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:34.495209 23018 ts_tablet_manager.cc:1905] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:34.502295 23018 ts_tablet_manager.cc:1918] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:34.503080 23017 tablet_replica.cc:330] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: stopping tablet replica
I20260811 12:10:34.503418 23017 raft_consensus.cc:2229] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:10:34.503867 23017 raft_consensus.cc:2258] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 12:10:34 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:10:34.507359 23017 ts_tablet_manager.cc:1905] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:10:34.510020 23020 ts_tablet_manager.cc:1918] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:34.510195 23020 log.cc:1196] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/16372ed3a96844e2a787612fdeff1146
I20260811 12:10:34.510759 23020 ts_tablet_manager.cc:1939] T 16372ed3a96844e2a787612fdeff1146 P 5fb200d349d64e22977546ea2a79581d: Deleting consensus metadata
I20260811 12:10:34.512619 22193 catalog_manager.cc:4841] TS 5fb200d349d64e22977546ea2a79581d (127.4.48.194:40373): tablet 16372ed3a96844e2a787612fdeff1146 (table test-table [id=e914a0a6c6c0476783c26829d903863c]) successfully deleted
I20260811 12:10:34.513020 23018 log.cc:1196] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/16372ed3a96844e2a787612fdeff1146
I20260811 12:10:34.513549 23018 ts_tablet_manager.cc:1939] T 16372ed3a96844e2a787612fdeff1146 P a8d61528cff941349a0078a3862255f3: Deleting consensus metadata
I20260811 12:10:34.513955 23017 ts_tablet_manager.cc:1918] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:10:34.514115 23017 log.cc:1196] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/16372ed3a96844e2a787612fdeff1146
I20260811 12:10:34.514643 23017 ts_tablet_manager.cc:1939] T 16372ed3a96844e2a787612fdeff1146 P 612f4958e507410ab58038108a32f425: Deleting consensus metadata
I20260811 12:10:34.515277 22192 catalog_manager.cc:4841] TS a8d61528cff941349a0078a3862255f3 (127.4.48.193:35417): tablet 16372ed3a96844e2a787612fdeff1146 (table test-table [id=e914a0a6c6c0476783c26829d903863c]) successfully deleted
I20260811 12:10:34.516249 22196 catalog_manager.cc:4841] TS 612f4958e507410ab58038108a32f425 (127.4.48.195:41515): tablet 16372ed3a96844e2a787612fdeff1146 (table test-table [id=e914a0a6c6c0476783c26829d903863c]) successfully deleted
I20260811 12:10:34.516680 22975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.481918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.4.48.193:35417)
0811 12:10:34.482096 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.482103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.495148 (+ 13045us) server_negotiation.cc:407] Connection header received
0811 12:10:34.495594 (+ 446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.495601 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.495706 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.495790 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:34.497731 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.498984 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.501056 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.501751 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.501855 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.508755 (+ 6900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:34.508770 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:34.508787 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:34.508826 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:34.512664 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.513742 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.513751 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.513755 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.513928 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.514548 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.514553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.514556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.515097 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:34.515463 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.516073 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.516304 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 12:10:34.517279 22507 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:34.611349 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 22311
I20260811 12:10:34.643003 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 22528
Aug 11 12:10:34 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:10:34.658428 22971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.644085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.4.48.194:40373)
0811 12:10:34.644276 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.644281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.644297 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:10:34.644347 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.644350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.644446 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.644536 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:34.645734 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.646676 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.647981 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.648416 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.648531 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.652635 (+ 4104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:34.652647 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:34.652665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:34.652701 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:34.655530 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.656298 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.656306 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.656308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.656465 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.656835 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.656840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.656843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.657272 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:34.657581 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.657887 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.658103 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:10:34.659161 22739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:34.770192 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 22528
I20260811 12:10:34.797699 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 22755
Aug 11 12:10:34 dist-test-slave-nj21 krb5kdc[22143](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:10:34.811630 22972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:10:34.798468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34128 (local address 127.4.48.195:41515)
0811 12:10:34.798669 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:10:34.798675 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:10:34.798763 (+ 88us) server_negotiation.cc:407] Connection header received
0811 12:10:34.798960 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:10:34.798965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:10:34.799035 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:10:34.799122 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:10:34.800493 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.801193 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.802454 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:10:34.802846 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:10:34.802957 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:10:34.806980 (+ 4023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:10:34.806994 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:10:34.807011 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:10:34.807049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:10:34.809467 (+ 2418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.810039 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.810046 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.810048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.810186 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:10:34.810477 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:10:34.810480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:10:34.810482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:10:34.810806 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:10:34.811026 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:10:34.811227 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 12:10:34.811381 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:10:34.812282 22947 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:34.898155 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 22755
I20260811 12:10:34.925240 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 22163
I20260811 12:10:34.926455 22265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:10:35.036016 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 22163
2026-08-11T12:10:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4510 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:10:35 dist-test-slave-nj21 krb5kdc[23038](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:10:35 dist-test-slave-nj21 krb5kdc[23038](info): set up 2 sockets
Aug 11 12:10:35 dist-test-slave-nj21 krb5kdc[23038](info): commencing operation
krb5kdc: starting...
W20260811 12:10:36.282398 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.005s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:10:36 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450236, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:10:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:10:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangeradmin_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangeradmin_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangerlookup_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangerlookup_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
I20260811 12:10:36.617954 4291 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXBhqNr/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/postgres -l logfile start
2026-08-11 12:10:38.922 UTC [23079] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:10:38.922 UTC [23079] LOG: listening on IPv4 address "127.4.48.212", port 60901
2026-08-11 12:10:38.925 UTC [23079] LOG: listening on Unix socket "/tmp/.s.PGSQL.60901"
2026-08-11 12:10:38.947 UTC [23085] LOG: database system was shut down at 2026-08-11 12:10:37 UTC
2026-08-11 12:10:38.950 UTC [23079] LOG: database system is ready to accept connections
I20260811 12:10:40.041460 4291 mini_postgres.cc:96] Postgres bound to 60901
2026-08-11 12:10:40.065 UTC [23097] FATAL: database "slave" does not exist
127.4.48.212:60901 - accepting connections
I20260811 12:10:40.066625 4291 mini_ranger.cc:161] Starting Ranger...
I20260811 12:10:40.094686 4291 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:10:40.512566 4291 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:10:40.512703 4291 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin
2026-08-11 12:10:40,900 [I] DB FLAVOR :POSTGRES
2026-08-11 12:10:40,901 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:10:40,901 [I] Checking connection..
2026-08-11 12:10:40,901 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:10:41,161 [I] Checking connection passed.
2026-08-11 12:10:41,162 [I] --------- Verifying version history table ---------
2026-08-11 12:10:41,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:10:41,407 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:10:41,407 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:10:41,664 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:10:41,664 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:10:41,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:10:41.916 UTC [23198] WARNING: there is no transaction in progress
2026-08-11 12:10:41,922 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:10:41,922 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:10:42,179 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:10:42,179 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:10:42,179 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:10:42,415 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:10:42,646 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nj21.c.gcp-upstream.internal','N') ;"
2026-08-11 12:10:42,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:10:43,142 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:10:43,142 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:10:43,385 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:10:43,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:10:43,633 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:10:43,857 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:10:43,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:10:44.724 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.740 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.754 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.768 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.781 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.797 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.846 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.851 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.864 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.873 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.882 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.895 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.903 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.909 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.914 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:44.920 UTC [23370] WARNING: there is no transaction in progress
2026-08-11 12:10:45,475 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:10:45,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:10:45,709 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:10:45,709 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:10:45,964 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:10:45,965 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:10:46,208 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:10:46,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:10:46,460 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:10:46,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nj21.c.gcp-upstream.internal';"
2026-08-11 12:10:46,956 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:10:46,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:10:47,210 [I] DB_PATCHES have already been applied
I20260811 12:10:47.216106 4291 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXBhqNr/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:10:47.216197 4291 mini_ranger.cc:193] Using host: 127.4.48.212
I20260811 12:10:47.237308 4291 mini_ranger.cc:227] Ranger admin URL: http://127.4.48.212:49693
Aug 11, 2026 12:10:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:10:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:10:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:10:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:10:47 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:10:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:10:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:10:48.503146 4291 mini_ranger.cc:160] Time spent starting Ranger: real 8.437s user 0.002s sys 0.054s
Aug 11, 2026 12:11:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:11:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:11:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:11:08 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:11:09 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:11:09.869066 4291 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangerkms_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/rangerkms_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.48.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.48.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:11:10 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450270, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:11:10.061393 4291 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:11:10.084558 4291 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:11:10.837872 4291 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:11:10.838035 4291 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms
I20260811 12:11:10.838146 4291 mini_ranger_kms.cc:227] Using postgres at 127.4.48.212:60901
2026-08-11 12:11:11,363 [I] DB FLAVOR :POSTGRES
2026-08-11 12:11:11,363 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:11:11,363 [I] Checking connection
2026-08-11 12:11:11,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:11:11,612 [I] connection success
2026-08-11 12:11:11,612 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:11:11,612 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:11:11,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:11:11,856 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:11:11,856 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:11:11,857 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:11:11,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60901/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:11:12,114 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:11:12.561583 4291 mini_ranger_kms.cc:310] Created kms service
I20260811 12:11:12.720628 4291 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:11:12.763607 4291 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:11:12 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450270, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:13.141047 4291 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:11:13.142338 4291 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:11:13.142401 4291 mini_ranger_kms.cc:240] Using host: 127.4.48.212
I20260811 12:11:13.171741 4291 mini_ranger_kms.cc:276] Ranger KMS PID: 23671
I20260811 12:11:13.171895 4291 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.48.212:58869
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:11:14 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:11:15 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:11:15.895787 4291 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.834s user 0.153s sys 0.355s
I20260811 12:11:15.896186 4291 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:11:15.896438 4291 mini_ranger_kms.cc:401] 127.4.48.212:58869/kms/v1/keys
Aug 11 12:11:15 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
Aug 11 12:11:16 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450275, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
Aug 11 12:11:18 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:11:18.753089 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:41559
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:41559
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:58869/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:11:19.122231 23742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:11:19.122538 23742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:11:19.122720 23742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:11:19.130039 23742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:11:19.130151 23742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:11:19.130239 23742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:11:19.130319 23742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:11:19.130390 23742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:11:19.141000 23742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.48.212:58869/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:41559
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:41559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:11:19.142319 23742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:11:19.143802 23742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:11:19.157752 23747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:19.157876 23749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:19.158406 23754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:11:19.158703 23742 server_base.cc:1034] running on GCE node
I20260811 12:11:19.159456 23742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:11:19.160884 23742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:11:19.162109 23742 hybrid_clock.cc:648] HybridClock initialized: now 1786450279162056 us; error 34 us; skew 500 ppm
I20260811 12:11:19.162500 23742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:11:19 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:11:19.169428 23742 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:11:19.171000 23742 webserver.cc:460] Webserver started at http://127.4.48.254:45431/ using document root <none> and password file <none>
I20260811 12:11:19.171651 23742 fs_manager.cc:359] Metadata directory not provided
I20260811 12:11:19.171823 23742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:11:19.172142 23742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:11:19 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450279, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:19.230252 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "043a23a5650540a195e92cd12ede1b06"
format_stamp: "Formatted at 2026-08-11 12:11:19 on dist-test-slave-nj21"
server_key: "fbbf94871206b507337e54aa31be87fb"
server_key_iv: "56f93616af376c04c655bda60a0374eb"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:19.231175 23742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "043a23a5650540a195e92cd12ede1b06"
format_stamp: "Formatted at 2026-08-11 12:11:19 on dist-test-slave-nj21"
server_key: "fbbf94871206b507337e54aa31be87fb"
server_key_iv: "56f93616af376c04c655bda60a0374eb"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:19.236816 23742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 12:11:19.253449 23769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:19.254866 23742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:11:19.255087 23742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "043a23a5650540a195e92cd12ede1b06"
format_stamp: "Formatted at 2026-08-11 12:11:19 on dist-test-slave-nj21"
server_key: "fbbf94871206b507337e54aa31be87fb"
server_key_iv: "56f93616af376c04c655bda60a0374eb"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:19.255300 23742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:11:19.272302 23742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:11:19.275723 23742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:11:19.275998 23742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:11:19.296090 23742 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:41559
I20260811 12:11:19.296109 23847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:41559 every 8 connection(s)
I20260811 12:11:19.297364 23742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:11:19.300895 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 23742
I20260811 12:11:19.301084 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:11:19.301287 23848 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:11:19.301755 4291 external_mini_cluster.cc:1428] Setting key d195bead382c9f2d19547e801b94add1
I20260811 12:11:19.315078 23848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: Bootstrap starting.
Aug 11 12:11:19 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:11:19.318903 23848 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: Neither blocks nor log segments found. Creating new log.
I20260811 12:11:19.320636 23848 log.cc:824] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: Log is configured to *not* fsync() on all Append() calls
I20260811 12:11:19.324118 23848 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: No bootstrap required, opened a new log
I20260811 12:11:19.327582 23852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:19.303626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.4.48.254:41559)
0811 12:11:19.304544 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:19.304555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:19.304600 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:11:19.307164 (+ 2564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:19.307191 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:19.307604 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:19.307999 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:19.309661 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:19.311031 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:19.312773 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:19.313470 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:19.313595 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:19.318220 (+ 4625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:19.318228 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:19.318242 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:19.318277 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:19.321382 (+ 3105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:19.323784 (+ 2402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:19.323792 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:19.323800 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:19.324004 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:19.324387 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:19.324403 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:19.324406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:19.325110 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:19.325384 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:19.325764 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:11:19.326171 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":431,"threads_started":1}
I20260811 12:11:19.333247 23848 raft_consensus.cc:348] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } }
I20260811 12:11:19.333596 23848 raft_consensus.cc:374] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:11:19.333679 23848 raft_consensus.cc:729] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 043a23a5650540a195e92cd12ede1b06, State: Initialized, Role: FOLLOWER
I20260811 12:11:19.334187 23848 consensus_queue.cc:260] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } }
I20260811 12:11:19.334416 23848 raft_consensus.cc:388] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:11:19.334533 23848 raft_consensus.cc:482] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:11:19.334661 23848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:11:19.337037 23848 raft_consensus.cc:504] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } }
I20260811 12:11:19.337620 23848 leader_election.cc:302] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 043a23a5650540a195e92cd12ede1b06; no voters:
I20260811 12:11:19.338451 23848 leader_election.cc:288] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:11:19.338522 23856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:11:19.339898 23856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [term 1 LEADER]: Becoming Leader. State: Replica: 043a23a5650540a195e92cd12ede1b06, State: Running, Role: LEADER
I20260811 12:11:19.340485 23856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } }
I20260811 12:11:19.341029 23848 sys_catalog.cc:564] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:11:19.343510 23858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "043a23a5650540a195e92cd12ede1b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } } }
I20260811 12:11:19.344110 23858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [sys.catalog]: This master's current role is: LEADER
I20260811 12:11:19.343555 23861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 043a23a5650540a195e92cd12ede1b06. Latest consensus state: current_term: 1 leader_uuid: "043a23a5650540a195e92cd12ede1b06" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "043a23a5650540a195e92cd12ede1b06" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41559 } } }
I20260811 12:11:19.344314 23861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06 [sys.catalog]: This master's current role is: LEADER
I20260811 12:11:19.344992 23872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:11:19.350309 23872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:11:19.351181 23848 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:11:19.368888 23872 catalog_manager.cc:1347] Generated new cluster ID: a1610fbc55e64c0198ffcc757a2a8f28
I20260811 12:11:19.369025 23872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:11:19.389204 23872 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:11:19.391218 23872 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:11:19.401908 23872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: Generated new TSK 0
I20260811 12:11:19.403016 23872 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:11:20.159781 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:58869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:11:20 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:11:20.599150 23925 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:11:20.599538 23925 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:11:20.599803 23925 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:11:20.611974 23925 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:11:20.612200 23925 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:11:20.612500 23925 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:11:20.628996 23925 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.48.212:58869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:11:20.630892 23925 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:11:20.632391 23925 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:11:20.655393 23943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:20.657804 23946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:20.658844 23950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:11:20.659905 23925 server_base.cc:1034] running on GCE node
I20260811 12:11:20.660573 23925 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:11:20.661571 23925 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:11:20.662817 23925 hybrid_clock.cc:648] HybridClock initialized: now 1786450280662739 us; error 53 us; skew 500 ppm
I20260811 12:11:20.663172 23925 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:11:20 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:11:20.666482 23925 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:11:20.668267 23925 webserver.cc:460] Webserver started at http://127.4.48.193:40349/ using document root <none> and password file <none>
I20260811 12:11:20.669018 23925 fs_manager.cc:359] Metadata directory not provided
I20260811 12:11:20.669145 23925 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:11:20.669499 23925 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:11:20 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:20.693157 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "b45b101711804e16aac22a4748b524b5"
format_stamp: "Formatted at 2026-08-11 12:11:20 on dist-test-slave-nj21"
server_key: "3ce881e9bc1b9c80407d15e65d51e842"
server_key_iv: "2397bd96cfcf2c91cd84aeda5d3e2a63"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:20.694459 23925 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "b45b101711804e16aac22a4748b524b5"
format_stamp: "Formatted at 2026-08-11 12:11:20 on dist-test-slave-nj21"
server_key: "3ce881e9bc1b9c80407d15e65d51e842"
server_key_iv: "2397bd96cfcf2c91cd84aeda5d3e2a63"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:20.705165 23925 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.008s sys 0.000s
I20260811 12:11:20.748452 23959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:20.753784 23925 fs_manager.cc:725] Time spent opening block manager: real 0.025s user 0.009s sys 0.008s
I20260811 12:11:20.754076 23925 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "b45b101711804e16aac22a4748b524b5"
format_stamp: "Formatted at 2026-08-11 12:11:20 on dist-test-slave-nj21"
server_key: "3ce881e9bc1b9c80407d15e65d51e842"
server_key_iv: "2397bd96cfcf2c91cd84aeda5d3e2a63"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:20.754467 23925 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:11:20.829841 23925 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:11:20.835721 23925 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:11:20.836133 23925 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:11:20.837504 23925 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:11:20.839421 23925 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:11:20.839530 23925 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:20.839699 23925 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:11:20.839789 23925 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
Aug 11 12:11:20 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:20.957376 23925 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:35463
I20260811 12:11:20.957374 24138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:35463 every 8 connection(s)
I20260811 12:11:20.959197 23925 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:11:20.966007 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 23925
I20260811 12:11:20.966225 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:11:20.967070 4291 external_mini_cluster.cc:1428] Setting key 16c2abc39631b6aa6a573fcc777bc268
Aug 11 12:11:20 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
I20260811 12:11:21.010089 24144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:20.962577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:59507 (local address 127.4.48.254:41559)
0811 12:11:20.986839 (+ 24262us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:20.986864 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:20.986887 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:11:20.986949 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:20.986954 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:20.987046 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:20.987153 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:20.989357 (+ 2204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:20.990173 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:20.995788 (+ 5615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:20.996397 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:20.996509 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:21.000844 (+ 4335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:21.000851 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:21.000855 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:21.000895 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:21.004436 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:21.005319 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:21.005325 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:21.005328 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:21.005516 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:21.008048 (+ 2532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:21.008053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:21.008057 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:21.008433 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:21.008582 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:21.009434 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 12:11:21.009687 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24163,"thread_start_us":24046,"threads_started":1}
I20260811 12:11:21.011263 24145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:20.963312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41559 (local address 127.4.48.193:59507)
0811 12:11:20.982480 (+ 19168us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:20.982555 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:20.985212 (+ 2657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:20.987621 (+ 2409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:20.987635 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:20.988286 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:11:20.989140 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:20.989166 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:20.990337 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:20.990342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:20.992215 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:20.992239 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:20.996799 (+ 4560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:20.996806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:20.997076 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:20.997859 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:11:20.997887 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:11:21.000596 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:11:21.004599 (+ 4003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:21.004607 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:21.004624 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:21.005153 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:21.007638 (+ 2485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:21.007643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:21.007646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:21.007908 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:21.008573 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:11:21.008581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:11:21.008991 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 12:11:21.009294 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:11:21.009948 (+ 654us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18794,"thread_start_us":22218,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
I20260811 12:11:21.013101 24140 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41559
I20260811 12:11:21.013448 24140 heartbeater.cc:464] Registering TS with master...
I20260811 12:11:21.014303 24140 heartbeater.cc:511] Master 127.4.48.254:41559 requested a full tablet report, sending...
I20260811 12:11:21.016979 23793 ts_manager.cc:194] Registered new tserver with Master: b45b101711804e16aac22a4748b524b5 (127.4.48.193:35463)
I20260811 12:11:21.019212 23793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:59507
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:11:21.048219 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:58869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:11:21 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450280, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
W20260811 12:11:21.554961 24151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:11:21.555313 24151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:11:21.555514 24151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:11:21.563067 24151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:11:21.563180 24151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:11:21.563342 24151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:11:21.573943 24151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.48.212:58869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:11:21.575205 24151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:11:21.576512 24151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:11:21.589421 24174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:21.589470 24175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:21.589377 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:11:21.590762 24151 server_base.cc:1034] running on GCE node
I20260811 12:11:21.591287 24151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:11:21.592173 24151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:11:21.593384 24151 hybrid_clock.cc:648] HybridClock initialized: now 1786450281593332 us; error 38 us; skew 500 ppm
I20260811 12:11:21.593701 24151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:11:21 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:11:21.596781 24151 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:11:21.598253 24151 webserver.cc:460] Webserver started at http://127.4.48.194:45509/ using document root <none> and password file <none>
I20260811 12:11:21.598857 24151 fs_manager.cc:359] Metadata directory not provided
I20260811 12:11:21.598971 24151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:11:21.599309 24151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:11:21 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:21.615518 24151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "1e9df20a14384b01a20399997d4a5fd6"
format_stamp: "Formatted at 2026-08-11 12:11:21 on dist-test-slave-nj21"
server_key: "aebb6e7013c2b16bc301fb79bee47003"
server_key_iv: "b5b80382ed4e9943122949cb5cdda114"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:21.616474 24151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "1e9df20a14384b01a20399997d4a5fd6"
format_stamp: "Formatted at 2026-08-11 12:11:21 on dist-test-slave-nj21"
server_key: "aebb6e7013c2b16bc301fb79bee47003"
server_key_iv: "b5b80382ed4e9943122949cb5cdda114"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:21.621645 24151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 12:11:21.635252 24188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:21.636662 24151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:11:21.636821 24151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "1e9df20a14384b01a20399997d4a5fd6"
format_stamp: "Formatted at 2026-08-11 12:11:21 on dist-test-slave-nj21"
server_key: "aebb6e7013c2b16bc301fb79bee47003"
server_key_iv: "b5b80382ed4e9943122949cb5cdda114"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:21.637032 24151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:11:21.651822 24151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:11:21.655470 24151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:11:21.655851 24151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:11:21.656800 24151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:11:21.658239 24151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:11:21.658320 24151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:21.658450 24151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:11:21.658519 24151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:21.706588 24151 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:35539
I20260811 12:11:21.706691 24355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:35539 every 8 connection(s)
I20260811 12:11:21.708374 24151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:11:21.714745 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 24151
I20260811 12:11:21.714926 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:11:21.715499 4291 external_mini_cluster.cc:1428] Setting key 8491445a39e89b41e92bd15394ce5a29
Aug 11 12:11:21 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450281, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:11:21.731745 24360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:21.713128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:38677 (local address 127.4.48.254:41559)
0811 12:11:21.713585 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:21.713590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:21.715075 (+ 1485us) server_negotiation.cc:407] Connection header received
0811 12:11:21.717627 (+ 2552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:21.717632 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:21.717730 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:21.717825 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:21.719258 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:21.720082 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:21.721581 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:21.722029 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:21.722135 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:21.725741 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:21.725747 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:21.725751 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:21.725792 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:21.728713 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:21.729383 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:21.729389 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:21.729392 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:21.729548 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:21.729981 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:21.729986 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:21.729990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:21.730280 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:21.730429 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:21.731078 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 12:11:21.731321 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":242,"threads_started":1}
I20260811 12:11:21.732686 24362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:21.713722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41559 (local address 127.4.48.194:38677)
0811 12:11:21.714903 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:21.714961 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:21.717359 (+ 2398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:21.718065 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:21.718073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:21.718519 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:11:21.719094 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:21.719112 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:21.720230 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:21.720234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:21.721447 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:21.721460 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:21.722239 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:21.722248 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:21.722477 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:21.723145 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:11:21.723167 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:11:21.725527 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:11:21.728842 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:21.728851 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:21.728867 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:21.729271 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:21.729671 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:21.729676 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:21.729680 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:21.729872 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:21.730412 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:11:21.730419 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:11:21.730718 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 12:11:21.730953 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 12:11:21.731480 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"thread_start_us":510,"threads_started":1}
I20260811 12:11:21.734270 24356 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41559
I20260811 12:11:21.734586 24356 heartbeater.cc:464] Registering TS with master...
I20260811 12:11:21.735486 24356 heartbeater.cc:511] Master 127.4.48.254:41559 requested a full tablet report, sending...
I20260811 12:11:21.737401 23793 ts_manager.cc:194] Registered new tserver with Master: 1e9df20a14384b01a20399997d4a5fd6 (127.4.48.194:35539)
I20260811 12:11:21.738372 23793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:38677
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:11:21.774482 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:58869/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:11:22.022979 24140 heartbeater.cc:503] Master 127.4.48.254:41559 was elected leader, sending a full tablet report...
W20260811 12:11:22.130338 24370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:11:22.130604 24370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:11:22.130739 24370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:11:22.137897 24370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:11:22.137995 24370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:11:22.138115 24370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:11:22.147815 24370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.48.212:58869/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41559
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:11:22.148983 24370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:11:22.150245 24370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:11:22.163168 24383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:22.163229 24377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:11:22.163239 24380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:11:22.164212 24370 server_base.cc:1034] running on GCE node
I20260811 12:11:22.164791 24370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:11:22.165638 24370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:11:22.166823 24370 hybrid_clock.cc:648] HybridClock initialized: now 1786450282166763 us; error 48 us; skew 500 ppm
I20260811 12:11:22.167101 24370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450282, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:11:22.170176 24370 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:11:22.171510 24370 webserver.cc:460] Webserver started at http://127.4.48.195:37165/ using document root <none> and password file <none>
I20260811 12:11:22.172175 24370 fs_manager.cc:359] Metadata directory not provided
I20260811 12:11:22.172281 24370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:11:22.172541 24370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450282, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:11:22.188601 24370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "a675fc405efb409cbe7896588670db73"
format_stamp: "Formatted at 2026-08-11 12:11:22 on dist-test-slave-nj21"
server_key: "fdf1cf66321a1f6dfe97fea19b4a5ef7"
server_key_iv: "c6e062cd6dac89f99395eb05ebf9fc40"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:22.189424 24370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "a675fc405efb409cbe7896588670db73"
format_stamp: "Formatted at 2026-08-11 12:11:22 on dist-test-slave-nj21"
server_key: "fdf1cf66321a1f6dfe97fea19b4a5ef7"
server_key_iv: "c6e062cd6dac89f99395eb05ebf9fc40"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:22.194947 24370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:11:22.208572 24394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:22.209986 24370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:11:22.210152 24370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "a675fc405efb409cbe7896588670db73"
format_stamp: "Formatted at 2026-08-11 12:11:22 on dist-test-slave-nj21"
server_key: "fdf1cf66321a1f6dfe97fea19b4a5ef7"
server_key_iv: "c6e062cd6dac89f99395eb05ebf9fc40"
server_key_version: "kuduclusterkey@0"
I20260811 12:11:22.210368 24370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:11:22.223014 24370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:11:22.226518 24370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:11:22.226788 24370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:11:22.227754 24370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:11:22.229065 24370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:11:22.229139 24370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:22.229254 24370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:11:22.229308 24370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:11:22.264549 24370 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:42167
I20260811 12:11:22.264567 24581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:42167 every 8 connection(s)
I20260811 12:11:22.265841 24370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:11:22.266281 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 24370
I20260811 12:11:22.266453 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:11:22.266986 4291 external_mini_cluster.cc:1428] Setting key d7dbe54c18303547d4bdd48bb16074dd
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450282, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:11:22.284999 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.268561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:36359 (local address 127.4.48.254:41559)
0811 12:11:22.269271 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.269276 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.270122 (+ 846us) server_negotiation.cc:407] Connection header received
0811 12:11:22.271923 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.271928 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.271997 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.272092 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:22.273368 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.274093 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.275552 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.276083 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.276187 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.279147 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:22.279153 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:22.279156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:22.279190 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:22.282300 (+ 3110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.282992 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.282997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.282999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.283136 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.283552 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.283556 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.283558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.283913 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:22.284037 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.284558 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.284736 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":243,"threads_started":1}
I20260811 12:11:22.285820 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.269091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41559 (local address 127.4.48.195:36359)
0811 12:11:22.269960 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.270006 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.271643 (+ 1637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.272254 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.272261 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.272678 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:11:22.273224 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.273240 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.274205 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.274208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.275441 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.275453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.276258 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.276263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.276462 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.277040 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:11:22.277060 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:11:22.278941 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:11:22.282433 (+ 3492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:22.282441 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:22.282454 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:22.282872 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:22.283256 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:11:22.283258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:11:22.283261 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:11:22.283454 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:11:22.284012 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:11:22.284017 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:11:22.284275 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.284489 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.284811 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":382,"threads_started":1}
I20260811 12:11:22.287065 24582 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41559
I20260811 12:11:22.287321 24582 heartbeater.cc:464] Registering TS with master...
I20260811 12:11:22.288007 24582 heartbeater.cc:511] Master 127.4.48.254:41559 requested a full tablet report, sending...
I20260811 12:11:22.289608 23793 ts_manager.cc:194] Registered new tserver with Master: a675fc405efb409cbe7896588670db73 (127.4.48.195:42167)
I20260811 12:11:22.290717 23793 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:36359
I20260811 12:11:22.295004 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:11:22.311976 24585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.299840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33138 (local address 127.4.48.254:41559)
0811 12:11:22.300007 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.300013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.300533 (+ 520us) server_negotiation.cc:407] Connection header received
0811 12:11:22.300678 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.300682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.300769 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.300851 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:22.302627 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.303386 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.304714 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.305161 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.305268 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.307392 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:22.307397 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:22.307401 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:22.307431 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:22.309813 (+ 2382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.310382 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.310385 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.310387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.310502 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.310780 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.310784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.310785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.311040 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:22.311221 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.311476 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.311632 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:11:22.315758 23793 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:11:22.322569 23793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:11:22.358987 24603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.334397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35463 (local address 127.0.0.1:35450)
0811 12:11:22.335347 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.335396 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.335582 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.350737 (+ 15155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.350745 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.350782 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:11:22.351419 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.351437 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.353301 (+ 1864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.353306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.355624 (+ 2318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.355643 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.357917 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.357923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.358113 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.358169 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.358280 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.358460 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":2,"thread_start_us":759,"threads_started":1}
I20260811 12:11:22.359505 24610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.335165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42167 (local address 127.0.0.1:35750)
0811 12:11:22.351385 (+ 16220us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.351417 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.351637 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.352421 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.352424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.352443 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:11:22.352858 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.352873 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.354366 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.354371 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.356402 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.356417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.358739 (+ 2322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.358747 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.358916 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.358939 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.359028 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.359143 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16087,"mutex_wait_us":19,"thread_start_us":16330,"threads_started":1}
I20260811 12:11:22.359689 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.335714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.4.48.195:42167)
0811 12:11:22.345169 (+ 9455us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.345179 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.351523 (+ 6344us) server_negotiation.cc:407] Connection header received
0811 12:11:22.351859 (+ 336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.351866 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.352076 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.352194 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:11:22.353077 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.354197 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.356562 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.357881 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.358379 (+ 498us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.358598 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.359081 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.359235 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8721,"thread_start_us":8552,"threads_started":1}
I20260811 12:11:22.359714 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.336192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42268 (local address 127.4.48.194:35539)
0811 12:11:22.350274 (+ 14082us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.350281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.352049 (+ 1768us) server_negotiation.cc:407] Connection header received
0811 12:11:22.352156 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.352163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.352371 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.352559 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:11:22.353144 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.354616 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.356753 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.358119 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.358248 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.359184 (+ 936us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.359255 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.359415 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13944,"thread_start_us":13848,"threads_started":1}
I20260811 12:11:22.360214 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.335915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:35539 (local address 127.0.0.1:42268)
0811 12:11:22.351797 (+ 15882us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.351819 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.351932 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.352560 (+ 628us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.352564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.352585 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:11:22.353007 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.353018 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.354785 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.354791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.356593 (+ 1802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.356607 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.358464 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.358469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.358633 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.358665 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.358774 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.358901 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15737,"mutex_wait_us":26,"thread_start_us":15165,"threads_started":1}
I20260811 12:11:22.361111 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.334731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35450 (local address 127.4.48.193:35463)
0811 12:11:22.350044 (+ 15313us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.350053 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.350086 (+ 33us) server_negotiation.cc:407] Connection header received
0811 12:11:22.350205 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.350214 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.350480 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.350699 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:11:22.351690 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.353149 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.355807 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.357179 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.357310 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.359766 (+ 2456us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.359863 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.360022 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15130,"thread_start_us":14962,"threads_started":1}
I20260811 12:11:22.363788 24268 tablet_service.cc:1467] Processing CreateTablet for tablet 7971cfc66ec94aeebb82f5edf000edea (DEFAULT_TABLE table=test-table [id=8a4a2f43d319468b9f6099526704c073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:11:22.363895 24473 tablet_service.cc:1467] Processing CreateTablet for tablet 7971cfc66ec94aeebb82f5edf000edea (DEFAULT_TABLE table=test-table [id=8a4a2f43d319468b9f6099526704c073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:11:22.365070 24268 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971cfc66ec94aeebb82f5edf000edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:11:22.364594 24035 tablet_service.cc:1467] Processing CreateTablet for tablet 7971cfc66ec94aeebb82f5edf000edea (DEFAULT_TABLE table=test-table [id=8a4a2f43d319468b9f6099526704c073]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:11:22.366106 24473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971cfc66ec94aeebb82f5edf000edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:11:22.366097 24035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7971cfc66ec94aeebb82f5edf000edea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:11:22.376379 24612 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Bootstrap starting.
I20260811 12:11:22.379336 24612 tablet_bootstrap.cc:654] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Neither blocks nor log segments found. Creating new log.
I20260811 12:11:22.380270 24614 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Bootstrap starting.
I20260811 12:11:22.381352 24615 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Bootstrap starting.
I20260811 12:11:22.381840 24612 log.cc:824] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:11:22.383903 24614 tablet_bootstrap.cc:654] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Neither blocks nor log segments found. Creating new log.
I20260811 12:11:22.384718 24615 tablet_bootstrap.cc:654] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Neither blocks nor log segments found. Creating new log.
I20260811 12:11:22.385239 24614 log.cc:824] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Log is configured to *not* fsync() on all Append() calls
I20260811 12:11:22.385313 24612 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: No bootstrap required, opened a new log
I20260811 12:11:22.385571 24612 ts_tablet_manager.cc:1397] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 12:11:22.386102 24615 log.cc:824] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:11:22.388441 24614 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: No bootstrap required, opened a new log
I20260811 12:11:22.388703 24614 ts_tablet_manager.cc:1397] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.006s
I20260811 12:11:22.390220 24615 tablet_bootstrap.cc:492] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: No bootstrap required, opened a new log
I20260811 12:11:22.390422 24615 ts_tablet_manager.cc:1397] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260811 12:11:22.396731 24612 raft_consensus.cc:348] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.397178 24612 raft_consensus.cc:374] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:11:22.397293 24612 raft_consensus.cc:729] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e9df20a14384b01a20399997d4a5fd6, State: Initialized, Role: FOLLOWER
I20260811 12:11:22.397876 24612 consensus_queue.cc:260] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.398057 24615 raft_consensus.cc:348] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.398471 24615 raft_consensus.cc:374] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:11:22.398582 24615 raft_consensus.cc:729] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45b101711804e16aac22a4748b524b5, State: Initialized, Role: FOLLOWER
I20260811 12:11:22.398699 24614 raft_consensus.cc:348] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.399118 24614 raft_consensus.cc:374] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:11:22.399226 24614 raft_consensus.cc:729] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a675fc405efb409cbe7896588670db73, State: Initialized, Role: FOLLOWER
I20260811 12:11:22.399492 24615 consensus_queue.cc:260] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.399895 24614 consensus_queue.cc:260] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.402611 24615 ts_tablet_manager.cc:1428] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 12:11:22.404812 24614 ts_tablet_manager.cc:1428] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Time spent starting tablet: real 0.016s user 0.014s sys 0.001s
I20260811 12:11:22.405206 24582 heartbeater.cc:503] Master 127.4.48.254:41559 was elected leader, sending a full tablet report...
I20260811 12:11:22.405776 24356 heartbeater.cc:503] Master 127.4.48.254:41559 was elected leader, sending a full tablet report...
I20260811 12:11:22.406380 24612 ts_tablet_manager.cc:1428] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Time spent starting tablet: real 0.021s user 0.013s sys 0.009s
W20260811 12:11:22.460398 24357 tablet.cc:2369] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:11:22.482120 24142 tablet.cc:2369] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:11:22.499936 24625 raft_consensus.cc:482] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:11:22.500238 24625 raft_consensus.cc:504] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.502056 24625 leader_election.cc:288] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b45b101711804e16aac22a4748b524b5 (127.4.48.193:35463), a675fc405efb409cbe7896588670db73 (127.4.48.195:42167)
I20260811 12:11:22.510805 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.502612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:54837 (local address 127.4.48.195:42167)
0811 12:11:22.502804 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.502811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.503513 (+ 702us) server_negotiation.cc:407] Connection header received
0811 12:11:22.503708 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.503713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.503795 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.503901 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:11:22.504671 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.505809 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.508423 (+ 2614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.509682 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.509850 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.509928 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.510440 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.510562 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 12:11:22.510809 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.502574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:40543 (local address 127.4.48.193:35463)
0811 12:11:22.502808 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.502813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.503506 (+ 693us) server_negotiation.cc:407] Connection header received
0811 12:11:22.503733 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.503737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.503805 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.503904 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:11:22.504669 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.505804 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.508409 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.509722 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.509872 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.509934 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.510440 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.510562 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 12:11:22.511089 24633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.502295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:35463 (local address 127.4.48.194:40543)
0811 12:11:22.503375 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.503398 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.503591 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.503951 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.503957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.504004 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:11:22.504523 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.504533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.505939 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.505943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.508271 (+ 2328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.508284 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.509934 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.509941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.510226 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.510318 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.510416 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.510614 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":938,"mutex_wait_us":196,"spinlock_wait_cycles":45696,"thread_start_us":712,"threads_started":1}
I20260811 12:11:22.511376 24631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.502421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42167 (local address 127.4.48.194:54837)
0811 12:11:22.503410 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0811 12:11:22.503425 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:11:22.503549 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:11:22.503951 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:11:22.503957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:11:22.504006 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:11:22.504511 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.504524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.505936 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.505941 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.508268 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:11:22.508284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.509934 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.509941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:11:22.510224 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.510317 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 12:11:22.510411 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:11:22.510614 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":885,"mutex_wait_us":142,"spinlock_wait_cycles":23040,"thread_start_us":570,"threads_started":1}
I20260811 12:11:22.512212 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971cfc66ec94aeebb82f5edf000edea" candidate_uuid: "1e9df20a14384b01a20399997d4a5fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45b101711804e16aac22a4748b524b5" is_pre_election: true
I20260811 12:11:22.512450 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971cfc66ec94aeebb82f5edf000edea" candidate_uuid: "1e9df20a14384b01a20399997d4a5fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a675fc405efb409cbe7896588670db73" is_pre_election: true
I20260811 12:11:22.513011 24078 raft_consensus.cc:2454] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e9df20a14384b01a20399997d4a5fd6 in term 0.
I20260811 12:11:22.513008 24500 raft_consensus.cc:2454] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e9df20a14384b01a20399997d4a5fd6 in term 0.
I20260811 12:11:22.513705 24192 leader_election.cc:302] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e9df20a14384b01a20399997d4a5fd6, b45b101711804e16aac22a4748b524b5; no voters:
I20260811 12:11:22.514135 24625 raft_consensus.cc:2781] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:11:22.514292 24625 raft_consensus.cc:482] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:11:22.514407 24625 raft_consensus.cc:3037] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:11:22.516742 24625 raft_consensus.cc:504] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.517374 24625 leader_election.cc:288] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [CANDIDATE]: Term 1 election: Requested vote from peers b45b101711804e16aac22a4748b524b5 (127.4.48.193:35463), a675fc405efb409cbe7896588670db73 (127.4.48.195:42167)
W20260811 12:11:22.517553 24584 tablet.cc:2369] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:11:22.517789 24078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971cfc66ec94aeebb82f5edf000edea" candidate_uuid: "1e9df20a14384b01a20399997d4a5fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b45b101711804e16aac22a4748b524b5"
I20260811 12:11:22.517868 24500 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7971cfc66ec94aeebb82f5edf000edea" candidate_uuid: "1e9df20a14384b01a20399997d4a5fd6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a675fc405efb409cbe7896588670db73"
I20260811 12:11:22.518018 24078 raft_consensus.cc:3037] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:11:22.518049 24500 raft_consensus.cc:3037] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:11:22.521144 24500 raft_consensus.cc:2454] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e9df20a14384b01a20399997d4a5fd6 in term 1.
I20260811 12:11:22.521144 24078 raft_consensus.cc:2454] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e9df20a14384b01a20399997d4a5fd6 in term 1.
I20260811 12:11:22.521592 24191 leader_election.cc:302] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e9df20a14384b01a20399997d4a5fd6, a675fc405efb409cbe7896588670db73; no voters:
I20260811 12:11:22.521896 24625 raft_consensus.cc:2781] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:11:22.522408 24625 raft_consensus.cc:686] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 1 LEADER]: Becoming Leader. State: Replica: 1e9df20a14384b01a20399997d4a5fd6, State: Running, Role: LEADER
I20260811 12:11:22.523021 24625 consensus_queue.cc:237] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } }
I20260811 12:11:22.527741 23793 catalog_manager.cc:5495] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e9df20a14384b01a20399997d4a5fd6 (127.4.48.194). New cstate: current_term: 1 leader_uuid: "1e9df20a14384b01a20399997d4a5fd6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1e9df20a14384b01a20399997d4a5fd6" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 35539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 } health_report { overall_health: UNKNOWN } } }
I20260811 12:11:22.549659 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.544340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42278 (local address 127.4.48.194:35539)
0811 12:11:22.544510 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.544514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.544530 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:11:22.544609 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.544612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.544689 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.544803 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:11:22.545188 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.545913 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.547589 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.548054 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.548144 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.548702 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:11:22.549168 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:11:22.549277 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.549359 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.549419 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 12:11:22.557197 24500 raft_consensus.cc:1261] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 1 FOLLOWER]: Refusing update from remote peer 1e9df20a14384b01a20399997d4a5fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:11:22.557226 24078 raft_consensus.cc:1261] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 1 FOLLOWER]: Refusing update from remote peer 1e9df20a14384b01a20399997d4a5fd6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:11:22.558079 24635 consensus_queue.cc:1035] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b45b101711804e16aac22a4748b524b5" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 35463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:11:22.558422 24625 consensus_queue.cc:1035] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a675fc405efb409cbe7896588670db73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:11:22.575870 24642 mvcc.cc:205] Tried to move back new op lower bound from 7317300357345390592 to 7317300357218762752. Current Snapshot: MvccSnapshot[applied={T|T < 7317300357345390592}]
I20260811 12:11:22.584653 23793 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:11:22.585000 23793 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:11:22.589576 23793 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 043a23a5650540a195e92cd12ede1b06: Sending DeleteTablet for 3 replicas of tablet 7971cfc66ec94aeebb82f5edf000edea
I20260811 12:11:22.591092 24473 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971cfc66ec94aeebb82f5edf000edea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:11:22 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:35750
I20260811 12:11:22.591403 24472 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971cfc66ec94aeebb82f5edf000edea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:11:22 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:35750
I20260811 12:11:22.592041 24268 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971cfc66ec94aeebb82f5edf000edea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:11:22 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42268
I20260811 12:11:22.593032 24035 tablet_service.cc:1514] Processing DeleteTablet for tablet 7971cfc66ec94aeebb82f5edf000edea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:11:22 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:35450
I20260811 12:11:22.593633 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 23925
I20260811 12:11:22.604609 24653 tablet_replica.cc:330] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: stopping tablet replica
I20260811 12:11:22.605029 24653 raft_consensus.cc:2229] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:11:22.605511 24653 raft_consensus.cc:2258] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:11:22.606055 24660 tablet_replica.cc:330] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: stopping tablet replica
I20260811 12:11:22.606421 24660 raft_consensus.cc:2229] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:11:22.606866 24660 raft_consensus.cc:2258] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:11:22.607656 24657 tablet_replica.cc:330] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: stopping tablet replica
I20260811 12:11:22.608057 24657 raft_consensus.cc:2229] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:11:22.608702 24657 raft_consensus.cc:2258] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:11:22.609364 24660 ts_tablet_manager.cc:1905] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:11:22.610754 24653 ts_tablet_manager.cc:1905] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:11:22.615043 24657 ts_tablet_manager.cc:1905] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:11:22.617316 24653 ts_tablet_manager.cc:1918] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:11:22.617477 24653 log.cc:1196] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/7971cfc66ec94aeebb82f5edf000edea
I20260811 12:11:22.617976 24653 ts_tablet_manager.cc:1939] T 7971cfc66ec94aeebb82f5edf000edea P a675fc405efb409cbe7896588670db73: Deleting consensus metadata
I20260811 12:11:22.619793 23773 catalog_manager.cc:4841] TS a675fc405efb409cbe7896588670db73 (127.4.48.195:42167): tablet 7971cfc66ec94aeebb82f5edf000edea (table test-table [id=8a4a2f43d319468b9f6099526704c073]) successfully deleted
W20260811 12:11:22.620812 23773 catalog_manager.cc:4808] TS a675fc405efb409cbe7896588670db73 (127.4.48.195:42167): delete failed for tablet 7971cfc66ec94aeebb82f5edf000edea because the tablet was not found. No further retry: Not found: Tablet not found: 7971cfc66ec94aeebb82f5edf000edea
I20260811 12:11:22.623857 24657 ts_tablet_manager.cc:1918] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:11:22.624029 24657 log.cc:1196] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/7971cfc66ec94aeebb82f5edf000edea
I20260811 12:11:22.624522 24657 ts_tablet_manager.cc:1939] T 7971cfc66ec94aeebb82f5edf000edea P 1e9df20a14384b01a20399997d4a5fd6: Deleting consensus metadata
I20260811 12:11:22.624979 24660 ts_tablet_manager.cc:1918] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:11:22.625142 24660 log.cc:1196] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/7971cfc66ec94aeebb82f5edf000edea
I20260811 12:11:22.625705 24660 ts_tablet_manager.cc:1939] T 7971cfc66ec94aeebb82f5edf000edea P b45b101711804e16aac22a4748b524b5: Deleting consensus metadata
I20260811 12:11:22.625762 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.596170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35456 (local address 127.4.48.193:35463)
0811 12:11:22.596335 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.596341 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.604612 (+ 8271us) server_negotiation.cc:407] Connection header received
0811 12:11:22.605472 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.605477 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.605569 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.605651 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:22.607837 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.608873 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.611506 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.612179 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.612274 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.618474 (+ 6200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:22.618489 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:22.618505 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:22.618554 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:22.622126 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.622991 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.623000 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.623003 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.623154 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.623527 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.623532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.623536 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.624055 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:22.624410 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.625019 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.625344 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:11:22.626158 23774 catalog_manager.cc:4841] TS 1e9df20a14384b01a20399997d4a5fd6 (127.4.48.194:35539): tablet 7971cfc66ec94aeebb82f5edf000edea (table test-table [id=8a4a2f43d319468b9f6099526704c073]) successfully deleted
I20260811 12:11:22.627413 23774 catalog_manager.cc:4841] TS b45b101711804e16aac22a4748b524b5 (127.4.48.193:35463): tablet 7971cfc66ec94aeebb82f5edf000edea (table test-table [id=8a4a2f43d319468b9f6099526704c073]) successfully deleted
I20260811 12:11:22.627475 24134 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:11:22.729092 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 23925
I20260811 12:11:22.752198 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 24151
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:11:22.769021 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.753067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.4.48.194:35539)
0811 12:11:22.753282 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.753288 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.753310 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:11:22.753374 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.753380 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.753465 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.753567 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:22.754793 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.756070 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.757284 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.757943 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.758073 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.762436 (+ 4363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:22.762452 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:22.762472 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:22.762522 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:22.766314 (+ 3792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.766918 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.766928 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.766933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.767109 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.767426 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.767432 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.767435 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.767912 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:22.768241 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.768434 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.768643 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:11:22.769836 24350 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:11:22.879961 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 24151
I20260811 12:11:22.903788 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 24370
Aug 11 12:11:22 dist-test-slave-nj21 krb5kdc[23038](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:11:22.917917 24607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:11:22.904483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.4.48.195:42167)
0811 12:11:22.904665 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:11:22.904669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:11:22.904703 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:11:22.904868 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:11:22.904872 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:11:22.904951 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:11:22.905032 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:11:22.906300 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.907001 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.908300 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:11:22.908682 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:11:22.908770 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:11:22.912797 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:11:22.912812 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:11:22.912829 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:11:22.912862 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:11:22.915531 (+ 2669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.916194 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.916202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.916204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.916341 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:11:22.916637 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:11:22.916639 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:11:22.916641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:11:22.916996 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:11:22.917205 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 12:11:22.917436 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 12:11:22.917595 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:11:22.918525 24574 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:11:23.005188 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 24370
I20260811 12:11:23.024574 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 23742
I20260811 12:11:23.025724 23841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:11:23.133975 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 23742
I20260811 12:11:23.164971 4291 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:11:23.509406 4291 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:11:23.509672 4291 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:11:23.853279 4291 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:11:23.853 UTC [23079] LOG: received smart shutdown request
2026-08-11 12:11:23.857 UTC [23079] LOG: background worker "logical replication launcher" (PID 23094) exited with exit code 1
2026-08-11 12:11:23.857 UTC [23086] LOG: shutting down
2026-08-11 12:11:23.987 UTC [23079] LOG: database system is shut down
2026-08-11T12:11:23Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49058 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:11:24 dist-test-slave-nj21 krb5kdc[24681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:11:24 dist-test-slave-nj21 krb5kdc[24681](info): set up 2 sockets
Aug 11 12:11:24 dist-test-slave-nj21 krb5kdc[24681](info): commencing operation
krb5kdc: starting...
W20260811 12:11:25.332688 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.001s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:11:25 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450285, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:11:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:11:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangeradmin_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangeradmin_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangerlookup_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangerlookup_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
I20260811 12:11:25.663398 4291 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskXBhqNr/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/postgres -l logfile start
2026-08-11 12:11:27.880 UTC [24731] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 12:11:27.880 UTC [24731] LOG: listening on IPv4 address "127.4.48.212", port 60727
2026-08-11 12:11:27.883 UTC [24731] LOG: listening on Unix socket "/tmp/.s.PGSQL.60727"
2026-08-11 12:11:27.904 UTC [24737] LOG: database system was shut down at 2026-08-11 12:11:26 UTC
2026-08-11 12:11:27.907 UTC [24731] LOG: database system is ready to accept connections
I20260811 12:11:29.033254 4291 mini_postgres.cc:96] Postgres bound to 60727
2026-08-11 12:11:29.055 UTC [24750] FATAL: database "slave" does not exist
127.4.48.212:60727 - accepting connections
I20260811 12:11:29.056161 4291 mini_ranger.cc:161] Starting Ranger...
I20260811 12:11:29.084342 4291 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 12:11:29.494227 4291 mini_ranger.cc:87] Created ranger Postgres database
I20260811 12:11:29.494338 4291 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin
2026-08-11 12:11:29,800 [I] DB FLAVOR :POSTGRES
2026-08-11 12:11:29,801 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:11:29,801 [I] Checking connection..
2026-08-11 12:11:29,801 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 12:11:30,058 [I] Checking connection passed.
2026-08-11 12:11:30,059 [I] --------- Verifying version history table ---------
2026-08-11 12:11:30,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:11:30,315 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:11:30,315 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:11:30,568 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 12:11:30,569 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 12:11:30,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 12:11:30.813 UTC [24849] WARNING: there is no transaction in progress
2026-08-11 12:11:30,821 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 12:11:30,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 12:11:31,066 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 12:11:31,066 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:11:31,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 12:11:31,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 12:11:31,578 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-nj21.c.gcp-upstream.internal','N') ;"
2026-08-11 12:11:31,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:11:32,070 [I] Table x_portal_user does not exist in database ranger
2026-08-11 12:11:32,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:11:32,305 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 12:11:32,306 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:11:32,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:11:32,788 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 12:11:32,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 12:11:33.683 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.697 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.711 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.724 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.738 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.754 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.810 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.816 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.828 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.837 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.846 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.859 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.867 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.873 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.879 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:33.885 UTC [25026] WARNING: there is no transaction in progress
2026-08-11 12:11:34,420 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 12:11:34,421 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 12:11:34,663 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 12:11:34,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 12:11:34,908 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 12:11:34,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 12:11:35,167 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 12:11:35,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 12:11:35,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 12:11:35,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-nj21.c.gcp-upstream.internal';"
2026-08-11 12:11:35,893 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 12:11:35,893 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 12:11:36,149 [I] DB_PATCHES have already been applied
I20260811 12:11:36.155520 4291 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskXBhqNr/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 12:11:36.155604 4291 mini_ranger.cc:193] Using host: 127.4.48.212
I20260811 12:11:36.174516 4291 mini_ranger.cc:227] Ranger admin URL: http://127.4.48.212:58599
Aug 11, 2026 12:11:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:11:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:11:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 12:11:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 12:11:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 12:11:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 12:11:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 12:11:37.471612 4291 mini_ranger.cc:160] Time spent starting Ranger: real 8.415s user 0.002s sys 0.050s
Aug 11, 2026 12:11:55 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 12:11:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 12:11:55 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 12:11:55 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 12:11:56 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 12:11:57.068265 4291 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.4.48.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.4.48.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangerkms_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.4.48.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/rangerkms_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.4.48.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.4.48.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.4.48.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/HTTP_127.4.48.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 12:11:57 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450317, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 12:11:57.278506 4291 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 12:11:57.302467 4291 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 12:11:58.058517 4291 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 12:11:58.058630 4291 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms
I20260811 12:11:58.058678 4291 mini_ranger_kms.cc:227] Using postgres at 127.4.48.212:60727
2026-08-11 12:11:58,396 [I] DB FLAVOR :POSTGRES
2026-08-11 12:11:58,396 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 12:11:58,396 [I] Checking connection
2026-08-11 12:11:58,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 12:11:58,653 [I] connection success
2026-08-11 12:11:58,653 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 12:11:58,653 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 12:11:58,653 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 12:11:58,939 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 12:11:58,940 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 12:11:58,940 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 12:11:58,940 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.4.48.212:60727/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 12:11:59,259 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 12:11:59.707381 4291 mini_ranger_kms.cc:310] Created kms service
I20260811 12:11:59.861091 4291 mini_ranger_kms.cc:326] Created kudu user
I20260811 12:11:59.900439 4291 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 12:11:59 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450317, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:00.299798 4291 mini_ranger_kms.cc:384] Added ranger policy
I20260811 12:12:00.300130 4291 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms:/tmp/dist-test-taskXBhqNr/build/asan/bin/postgresql.jar:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskXBhqNr/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 12:12:00.300201 4291 mini_ranger_kms.cc:240] Using host: 127.4.48.212
I20260811 12:12:00.326625 4291 mini_ranger_kms.cc:276] Ranger KMS PID: 25308
I20260811 12:12:00.326747 4291 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.4.48.212:57035
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 12:12:01 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 12:12:02 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450322, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:02.983376 4291 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.705s user 0.066s sys 0.308s
I20260811 12:12:02.983624 4291 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 12:12:02.983872 4291 mini_ranger_kms.cc:401] 127.4.48.212:57035/kms/v1/keys
Aug 11 12:12:03 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450322, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
Aug 11 12:12:03 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450322, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.4.48.212@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
Aug 11 12:12:05 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:12:05.794901 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42423
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:42423
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:06.122726 25379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:06.123009 25379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:06.123104 25379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:06.130432 25379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:06.130545 25379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:06.130601 25379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:06.130656 25379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:06.130703 25379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:06.140901 25379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
--ranger_kms_url=127.4.48.212:57035/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:42423
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:06.142206 25379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:06.143491 25379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:06.154834 25389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:06.154775 25388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:06.154841 25394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:06.156018 25379 server_base.cc:1034] running on GCE node
I20260811 12:12:06.156777 25379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:06.158162 25379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:06.159363 25379 hybrid_clock.cc:648] HybridClock initialized: now 1786450326159322 us; error 31 us; skew 500 ppm
I20260811 12:12:06.159740 25379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:06 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450326, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:06.162698 25379 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:06.164146 25379 webserver.cc:460] Webserver started at http://127.4.48.254:36715/ using document root <none> and password file <none>
I20260811 12:12:06.164783 25379 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:06.164896 25379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:06.165195 25379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:12:06 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450326, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:06.213068 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "7af9aa743bd34882ad22c831d58e8958"
format_stamp: "Formatted at 2026-08-11 12:12:06 on dist-test-slave-nj21"
server_key: "82ba3da08c8a2a1d3b1bc8b45e3d1416"
server_key_iv: "fbd946aa10f8327c1dd258e97ddfebb0"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:06.214007 25379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "7af9aa743bd34882ad22c831d58e8958"
format_stamp: "Formatted at 2026-08-11 12:12:06 on dist-test-slave-nj21"
server_key: "82ba3da08c8a2a1d3b1bc8b45e3d1416"
server_key_iv: "fbd946aa10f8327c1dd258e97ddfebb0"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:06.219525 25379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 12:12:06.236470 25408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:06.238161 25379 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:12:06.238395 25379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "7af9aa743bd34882ad22c831d58e8958"
format_stamp: "Formatted at 2026-08-11 12:12:06 on dist-test-slave-nj21"
server_key: "82ba3da08c8a2a1d3b1bc8b45e3d1416"
server_key_iv: "fbd946aa10f8327c1dd258e97ddfebb0"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:06.238660 25379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:06.268041 25379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:06.271946 25379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:06.272249 25379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:06.288283 25379 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:42423
I20260811 12:12:06.288312 25481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:42423 every 8 connection(s)
I20260811 12:12:06.289552 25379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:06.291200 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 25379
I20260811 12:12:06.291390 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:12:06.292104 4291 external_mini_cluster.cc:1428] Setting key a890178aa6a000371131e29e74173e3c
I20260811 12:12:06.293799 25482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:12:06 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:06.306699 25482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Bootstrap starting.
I20260811 12:12:06.310745 25482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:06.311932 25482 log.cc:824] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:06.314383 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:06.294037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58012 (local address 127.4.48.254:42423)
0811 12:12:06.294855 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:06.294867 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:06.294907 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:12:06.296441 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:06.296462 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:06.296819 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:06.297145 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:06.298742 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:06.300145 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:06.301886 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:06.302513 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:06.302630 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:06.306900 (+ 4270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:06.306911 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:06.306928 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:06.306978 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:06.310514 (+ 3536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:06.311175 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:06.311183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:06.311191 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:06.311392 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:06.311734 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:06.311739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:06.311742 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:06.312398 (+ 656us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:06.312685 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:06.313092 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 12:12:06.313397 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":369,"threads_started":1}
I20260811 12:12:06.314985 25482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: No bootstrap required, opened a new log
I20260811 12:12:06.322484 25482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:06.322805 25482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:06.322896 25482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af9aa743bd34882ad22c831d58e8958, State: Initialized, Role: FOLLOWER
I20260811 12:12:06.323414 25482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:06.323632 25482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:06.323772 25482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:06.323930 25482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:06.325968 25482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:06.326385 25482 leader_election.cc:302] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7af9aa743bd34882ad22c831d58e8958; no voters:
I20260811 12:12:06.326939 25482 leader_election.cc:288] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:12:06.327080 25488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:06.327868 25488 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 LEADER]: Becoming Leader. State: Replica: 7af9aa743bd34882ad22c831d58e8958, State: Running, Role: LEADER
I20260811 12:12:06.328387 25488 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:06.328887 25482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:06.332073 25490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7af9aa743bd34882ad22c831d58e8958. Latest consensus state: current_term: 1 leader_uuid: "7af9aa743bd34882ad22c831d58e8958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } } }
I20260811 12:12:06.332044 25489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7af9aa743bd34882ad22c831d58e8958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } } }
I20260811 12:12:06.332489 25490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:06.332487 25489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:06.333940 25504 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:06.338513 25504 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:06.342983 25482 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 12:12:06.366207 25504 catalog_manager.cc:1347] Generated new cluster ID: 2a399a3f7b354f24948a820d4bfec279
I20260811 12:12:06.366349 25504 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:06.375123 25504 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:12:06.376461 25504 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:06.390153 25504 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Generated new TSK 0
I20260811 12:12:06.390956 25504 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:12:07.155090 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:07 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 12:12:07.581480 25540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:07.581935 25540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:07.582139 25540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:07.593393 25540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:07.593547 25540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:07.593745 25540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:07.608937 25540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:07.610726 25540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:07.612562 25540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:07.630105 25553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:07.630133 25555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:07.645500 25559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:07.645893 25540 server_base.cc:1034] running on GCE node
I20260811 12:12:07.646744 25540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:07.650697 25540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:07.651991 25540 hybrid_clock.cc:648] HybridClock initialized: now 1786450327651891 us; error 86 us; skew 500 ppm
I20260811 12:12:07.652457 25540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:07 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:07.656625 25540 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:07.664484 25540 webserver.cc:460] Webserver started at http://127.4.48.193:33871/ using document root <none> and password file <none>
I20260811 12:12:07.665470 25540 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:07.665678 25540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:07.666097 25540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:12:07 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:07.695207 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "861ed28d44a94372818c4719abcb786e"
format_stamp: "Formatted at 2026-08-11 12:12:07 on dist-test-slave-nj21"
server_key: "76810d93957f99f4d5af96ff80483aca"
server_key_iv: "f42a9ef200036cd4053aaa010abcce3b"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:07.696295 25540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "861ed28d44a94372818c4719abcb786e"
format_stamp: "Formatted at 2026-08-11 12:12:07 on dist-test-slave-nj21"
server_key: "76810d93957f99f4d5af96ff80483aca"
server_key_iv: "f42a9ef200036cd4053aaa010abcce3b"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:07.703547 25540 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 12:12:07.722550 25570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:07.724149 25540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:12:07.724401 25540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "861ed28d44a94372818c4719abcb786e"
format_stamp: "Formatted at 2026-08-11 12:12:07 on dist-test-slave-nj21"
server_key: "76810d93957f99f4d5af96ff80483aca"
server_key_iv: "f42a9ef200036cd4053aaa010abcce3b"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:07.724774 25540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:07.757277 25540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:07.762627 25540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:07.763033 25540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:07.764276 25540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:07.766075 25540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:07.766219 25540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:07.766433 25540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:07.766506 25540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:07.844806 25540 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:36117
I20260811 12:12:07.844892 25738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:36117 every 8 connection(s)
I20260811 12:12:07.846449 25540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:07.854988 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 25540
I20260811 12:12:07.855212 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:12:07.856099 4291 external_mini_cluster.cc:1428] Setting key 5cab27b9bf55b3deff85bcd5aa6210e0
Aug 11 12:12:07 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
Aug 11 12:12:07 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:07.892210 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:07.855861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.193:39897)
0811 12:12:07.864898 (+ 9037us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:07.864962 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:07.868681 (+ 3719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:07.872323 (+ 3642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:07.872335 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:07.872978 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:07.873790 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:07.873816 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:07.875667 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:07.875693 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:07.877538 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:07.877552 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:07.878873 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:07.878879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:07.879133 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:07.879819 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:07.879854 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:07.882456 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:07.886897 (+ 4441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:07.886906 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:07.886922 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:07.888165 (+ 1243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:07.888608 (+ 443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:07.888613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:07.888617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:07.888850 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:07.889540 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:07.889548 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:07.889958 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 12:12:07.890238 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 12:12:07.890749 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8674,"thread_start_us":11676,"threads_started":1}
I20260811 12:12:07.893350 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:07.855133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:39897 (local address 127.4.48.254:42423)
0811 12:12:07.871736 (+ 16603us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:07.871743 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:07.871763 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:12:07.871821 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:07.871826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:07.871923 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:07.872046 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:07.874391 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:07.875500 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:07.877710 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:07.878472 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:07.878587 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:07.882717 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:07.882724 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:07.882728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:07.882766 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:07.886735 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:07.888303 (+ 1568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:07.888308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:07.888312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:07.888476 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:07.888988 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:07.888993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:07.888996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:07.889319 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:07.889468 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:07.892737 (+ 3269us) server_negotiation.cc:299] Negotiation successful
0811 12:12:07.892978 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2133,"server-negotiator.queue_time_us":16444,"thread_start_us":16285,"threads_started":1}
I20260811 12:12:07.895555 25739 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:07.895934 25739 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:07.896814 25739 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:07.899370 25437 ts_manager.cc:194] Registered new tserver with Master: 861ed28d44a94372818c4719abcb786e (127.4.48.193:36117)
I20260811 12:12:07.901468 25437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:39897
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:12:07.971237 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:08 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450327, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
W20260811 12:12:08.476063 25747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:08.476490 25747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:08.476649 25747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:08.488206 25747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:08.488365 25747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:08.488574 25747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:08.503911 25747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:08.505282 25747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:08.506651 25747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:08.523393 25763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:08.523486 25775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:08.524005 25762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:08.524806 25747 server_base.cc:1034] running on GCE node
I20260811 12:12:08.525390 25747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:08.526350 25747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:08.527567 25747 hybrid_clock.cc:648] HybridClock initialized: now 1786450328527510 us; error 51 us; skew 500 ppm
I20260811 12:12:08.527901 25747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:08 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:08.535424 25747 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:08.536942 25747 webserver.cc:460] Webserver started at http://127.4.48.194:37683/ using document root <none> and password file <none>
I20260811 12:12:08.537587 25747 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:08.537698 25747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:08.538002 25747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:12:08 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:08.554183 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "d6df9675b4294503a68aa08310cadb8d"
format_stamp: "Formatted at 2026-08-11 12:12:08 on dist-test-slave-nj21"
server_key: "7a592efdedcee9d2001ffb024a275f9b"
server_key_iv: "26eb1fef90d55d317257a178796b3c01"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:08.555016 25747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "d6df9675b4294503a68aa08310cadb8d"
format_stamp: "Formatted at 2026-08-11 12:12:08 on dist-test-slave-nj21"
server_key: "7a592efdedcee9d2001ffb024a275f9b"
server_key_iv: "26eb1fef90d55d317257a178796b3c01"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:08.560563 25747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 12:12:08.578099 25786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:08.579921 25747 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:12:08.580109 25747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "d6df9675b4294503a68aa08310cadb8d"
format_stamp: "Formatted at 2026-08-11 12:12:08 on dist-test-slave-nj21"
server_key: "7a592efdedcee9d2001ffb024a275f9b"
server_key_iv: "26eb1fef90d55d317257a178796b3c01"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:08.580364 25747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:08.617024 25747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:08.620720 25747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:08.620985 25747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:08.621923 25747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:08.623273 25747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:08.623347 25747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:08.623442 25747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:08.623484 25747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:08.653662 25747 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:36477
I20260811 12:12:08.653697 25966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:36477 every 8 connection(s)
I20260811 12:12:08.654906 25747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:08.661002 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 25747
I20260811 12:12:08.661181 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:12:08.661667 4291 external_mini_cluster.cc:1428] Setting key 507304d7c7e4c3f82a35d128600d75b1
Aug 11 12:12:08 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450328, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:08.674211 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:08.657681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:47677 (local address 127.4.48.254:42423)
0811 12:12:08.658211 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:08.658216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:08.659214 (+ 998us) server_negotiation.cc:407] Connection header received
0811 12:12:08.660915 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:08.660921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:08.661000 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:08.661102 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:08.662574 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:08.663333 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:08.664896 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:08.665294 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:08.665417 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:08.668593 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:08.668598 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:08.668601 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:08.668632 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:08.671173 (+ 2541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:08.671927 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:08.671932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:08.671935 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:08.672079 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:08.672500 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:08.672503 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:08.672506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:08.672852 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:08.672993 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:08.673606 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 12:12:08.673849 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":258,"threads_started":1}
I20260811 12:12:08.675108 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:08.658274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.194:47677)
0811 12:12:08.659048 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:08.659096 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:08.660636 (+ 1540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:08.661374 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:08.661382 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:08.661821 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:08.662406 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:08.662424 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:08.663491 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:08.663495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:08.664754 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:08.664767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:08.665520 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:08.665527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:08.665773 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:08.666372 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:08.666395 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:08.668392 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:08.671332 (+ 2940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:08.671341 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:08.671353 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:08.671796 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:08.672181 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:08.672185 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:08.672187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:08.672393 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:08.672952 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:08.672957 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:08.673237 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 12:12:08.673474 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 12:12:08.673951 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":312,"threads_started":1}
I20260811 12:12:08.676483 25967 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:08.676834 25967 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:08.677664 25967 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:08.679198 25437 ts_manager.cc:194] Registered new tserver with Master: d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477)
I20260811 12:12:08.680011 25437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:47677
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:12:08.721005 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:08.905149 25739 heartbeater.cc:503] Master 127.4.48.254:42423 was elected leader, sending a full tablet report...
W20260811 12:12:09.043130 25976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:09.043397 25976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:09.043540 25976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:09.050706 25976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:09.050817 25976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:09.050958 25976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:09.060644 25976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:09.061954 25976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:09.063252 25976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:09.076028 25987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:09.076072 25992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:09.076098 25988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:09.076937 25976 server_base.cc:1034] running on GCE node
I20260811 12:12:09.077495 25976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:09.078393 25976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:09.079591 25976 hybrid_clock.cc:648] HybridClock initialized: now 1786450329079543 us; error 34 us; skew 500 ppm
I20260811 12:12:09.079903 25976 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:09.082824 25976 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:09.084193 25976 webserver.cc:460] Webserver started at http://127.4.48.195:42771/ using document root <none> and password file <none>
I20260811 12:12:09.084822 25976 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:09.084939 25976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:09.085249 25976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:09.102154 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "040297a67d5447d58a4ca4a9f0e282c1"
format_stamp: "Formatted at 2026-08-11 12:12:09 on dist-test-slave-nj21"
server_key: "5432bd168a8b981bb1d6d535212fbd6e"
server_key_iv: "c9493a9468b3c8a1fedc57a99d0223ba"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:09.103211 25976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "040297a67d5447d58a4ca4a9f0e282c1"
format_stamp: "Formatted at 2026-08-11 12:12:09 on dist-test-slave-nj21"
server_key: "5432bd168a8b981bb1d6d535212fbd6e"
server_key_iv: "c9493a9468b3c8a1fedc57a99d0223ba"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:09.110045 25976 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.009s sys 0.000s
I20260811 12:12:09.128022 26004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:09.129889 25976 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:12:09.130084 25976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "040297a67d5447d58a4ca4a9f0e282c1"
format_stamp: "Formatted at 2026-08-11 12:12:09 on dist-test-slave-nj21"
server_key: "5432bd168a8b981bb1d6d535212fbd6e"
server_key_iv: "c9493a9468b3c8a1fedc57a99d0223ba"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:09.130342 25976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:09.147969 25976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:09.151768 25976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:09.152029 25976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:09.152894 25976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:09.154156 25976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:09.154229 25976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:09.154318 25976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:09.154364 25976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:09.185133 25976 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44081
I20260811 12:12:09.185173 26203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44081 every 8 connection(s)
I20260811 12:12:09.186502 25976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:09.195822 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 25976
I20260811 12:12:09.196002 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:12:09.196530 4291 external_mini_cluster.cc:1428] Setting key 7e18973ca0a1b2319bfcff1f0b059744
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450329, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:09.205701 26208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:09.189011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:45967 (local address 127.4.48.254:42423)
0811 12:12:09.189487 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:09.189492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:09.190455 (+ 963us) server_negotiation.cc:407] Connection header received
0811 12:12:09.192214 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:09.192219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:09.192286 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:09.192389 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:09.193710 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.194423 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.196197 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.196617 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.196739 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:09.200474 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:09.200479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:09.200482 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:09.200513 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:09.202923 (+ 2410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.203588 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.203591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.203594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.203767 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.204159 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.204162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.204164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.204459 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:09.204573 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:09.205219 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 12:12:09.205426 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":206,"threads_started":1}
I20260811 12:12:09.206534 26210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:09.189587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.195:45967)
0811 12:12:09.190302 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:09.190340 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:09.191923 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:09.192565 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:09.192573 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:09.192984 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:09.193538 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:09.193554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.194561 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.194565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:09.196059 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:09.196074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.196863 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.196871 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:09.197213 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:09.198082 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:09.198109 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:09.200276 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:09.203050 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:09.203057 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:09.203068 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:09.203491 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:09.203861 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:09.203864 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:09.203867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:09.204063 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:09.204550 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:09.204556 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:09.204856 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 12:12:09.205076 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 12:12:09.205460 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":300,"threads_started":1}
I20260811 12:12:09.207851 26204 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:09.208159 26204 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:09.208879 26204 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:09.210331 25437 ts_manager.cc:194] Registered new tserver with Master: 040297a67d5447d58a4ca4a9f0e282c1 (127.4.48.195:44081)
I20260811 12:12:09.211264 25437 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:45967
I20260811 12:12:09.211964 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:12:09.213603 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 25540
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:12:09.229868 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:09.214523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.4.48.193:36117)
0811 12:12:09.215154 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:09.215162 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:09.215185 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:12:09.215238 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:09.215243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:09.215422 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:09.215554 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:09.217104 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.218100 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.219479 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.220051 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.220165 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:09.224316 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:09.224327 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:09.224343 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:09.224377 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:09.227058 (+ 2681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.227747 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.227755 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.227758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.227913 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.228265 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.228268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.228271 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.228601 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:09.228923 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:09.229211 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 12:12:09.229486 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":289,"threads_started":1}
I20260811 12:12:09.231215 25733 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:09.311790 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 25540
I20260811 12:12:09.335651 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 25747
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:12:09.351936 26215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:09.336669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34970 (local address 127.4.48.194:36477)
0811 12:12:09.337265 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:09.337272 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:09.337298 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:12:09.337357 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:09.337364 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:09.337572 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:09.337701 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:09.339030 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.340146 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.341588 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.342147 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.342242 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:09.346682 (+ 4440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:09.346693 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:09.346707 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:09.346741 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:09.349589 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.350168 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.350174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.350177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.350318 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.350594 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.350597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.350599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.350925 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:09.351143 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:09.351371 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 12:12:09.351567 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":314,"threads_started":1}
I20260811 12:12:09.353204 25962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:09.433905 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 25747
I20260811 12:12:09.458746 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 25976
Aug 11 12:12:09 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:12:09.474716 26218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:09.459746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49136 (local address 127.4.48.195:44081)
0811 12:12:09.460403 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:09.460411 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:09.460439 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:12:09.460499 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:09.460505 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:09.460728 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:09.460861 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:09.462091 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.463093 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.464427 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:09.464988 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:09.465086 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:09.469160 (+ 4074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:09.469172 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:09.469186 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:09.469232 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:09.472015 (+ 2783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.472677 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.472684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.472686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.472823 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:09.473168 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:09.473172 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:09.473174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:09.473516 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:09.473830 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:09.474142 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 12:12:09.474406 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":323,"threads_started":1}
I20260811 12:12:09.475956 26194 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:09.556669 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 25976
I20260811 12:12:09.580435 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 25379
I20260811 12:12:09.581683 25476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:09.686893 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 25379
I20260811 12:12:09.720494 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42423
--webserver_interface=127.4.48.254
--webserver_port=36715
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:42423
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:10.043452 26224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:10.043737 26224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:10.043866 26224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:10.050951 26224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:10.051055 26224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:10.051129 26224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:10.051208 26224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:10.051272 26224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:10.061497 26224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:42423
--ranger_config_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:42423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=36715
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:10.062781 26224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:10.064152 26224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:10.075397 26233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:10.075361 26238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:10.075428 26234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:10.080936 26224 server_base.cc:1034] running on GCE node
I20260811 12:12:10.081647 26224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:10.083005 26224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:10.084208 26224 hybrid_clock.cc:648] HybridClock initialized: now 1786450330084157 us; error 42 us; skew 500 ppm
I20260811 12:12:10.084532 26224 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:10 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450330, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:10.087414 26224 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:10.088886 26224 webserver.cc:460] Webserver started at http://127.4.48.254:36715/ using document root <none> and password file <none>
I20260811 12:12:10.089522 26224 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:10.089640 26224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:12:10 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450330, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:10.136899 26224 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:12:10.140096 26248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:10.141525 26224 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:10.141698 26224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "7af9aa743bd34882ad22c831d58e8958"
format_stamp: "Formatted at 2026-08-11 12:12:06 on dist-test-slave-nj21"
server_key: "82ba3da08c8a2a1d3b1bc8b45e3d1416"
server_key_iv: "fbd946aa10f8327c1dd258e97ddfebb0"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:10.142655 26224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:10.160619 26224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:10.164491 26224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:10.164762 26224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:10.182549 26224 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:42423
I20260811 12:12:10.182586 26326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:42423 every 8 connection(s)
I20260811 12:12:10.184055 26224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:10.185873 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 26224
I20260811 12:12:10.186944 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:36117
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:42423
--webserver_port=33871
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:10.192884 26328 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:12:10.197050 26328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Bootstrap starting.
I20260811 12:12:10.215250 26328 log.cc:824] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:10.224342 26328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:12:10.224843 26328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Bootstrap complete.
I20260811 12:12:10.233533 26328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:10.234349 26328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af9aa743bd34882ad22c831d58e8958, State: Initialized, Role: FOLLOWER
I20260811 12:12:10.234972 26328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:10.235180 26328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:10.235301 26328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:10.235458 26328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:12:10.239183 26328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:10.239634 26328 leader_election.cc:302] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7af9aa743bd34882ad22c831d58e8958; no voters:
I20260811 12:12:10.240283 26328 leader_election.cc:288] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:12:10.240403 26334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:12:10.241540 26334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [term 2 LEADER]: Becoming Leader. State: Replica: 7af9aa743bd34882ad22c831d58e8958, State: Running, Role: LEADER
I20260811 12:12:10.242126 26334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } }
I20260811 12:12:10.242336 26328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:10.245546 26336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7af9aa743bd34882ad22c831d58e8958. Latest consensus state: current_term: 2 leader_uuid: "7af9aa743bd34882ad22c831d58e8958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } } }
I20260811 12:12:10.245920 26336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:10.246457 26335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7af9aa743bd34882ad22c831d58e8958" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af9aa743bd34882ad22c831d58e8958" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 42423 } } }
I20260811 12:12:10.246727 26335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:10.252668 26347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:10.262369 26347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:10.263821 26347 catalog_manager.cc:1259] Loaded cluster ID: 2a399a3f7b354f24948a820d4bfec279
I20260811 12:12:10.263978 26347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:10.268004 26347 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:10.269501 26347 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Loaded TSK: 0
I20260811 12:12:10.271118 26347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:12:10.275774 26328 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260811 12:12:10.632532 26332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:10.633052 26332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:10.633251 26332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:10.644649 26332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:10.644783 26332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:10.644951 26332 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:10.658661 26332 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:36117
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=33871
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:10.660681 26332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:10.662519 26332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:10.687136 26391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:10.688520 26392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:10.697098 26395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:10.697757 26332 server_base.cc:1034] running on GCE node
I20260811 12:12:10.698632 26332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:10.699980 26332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:10.701308 26332 hybrid_clock.cc:648] HybridClock initialized: now 1786450330701194 us; error 90 us; skew 500 ppm
I20260811 12:12:10.701720 26332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:10 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450330, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:10.705982 26332 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:10.712069 26332 webserver.cc:460] Webserver started at http://127.4.48.193:33871/ using document root <none> and password file <none>
I20260811 12:12:10.713080 26332 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:10.713236 26332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:12:10 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450330, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:10.754853 26332 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.004s sys 0.008s
I20260811 12:12:10.759843 26403 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:10.761722 26332 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 12:12:10.761976 26332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "861ed28d44a94372818c4719abcb786e"
format_stamp: "Formatted at 2026-08-11 12:12:07 on dist-test-slave-nj21"
server_key: "76810d93957f99f4d5af96ff80483aca"
server_key_iv: "f42a9ef200036cd4053aaa010abcce3b"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:10.763411 26332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:10.785301 26332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:10.790313 26332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:10.790640 26332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:10.791945 26332 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:10.793777 26332 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:10.793895 26332 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:10.794010 26332 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:10.794088 26332 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:10.860425 26332 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:36117
I20260811 12:12:10.860462 26585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:36117 every 8 connection(s)
I20260811 12:12:10.862113 26332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:10.865687 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 26332
I20260811 12:12:10.866432 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:36477
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:42423
--webserver_port=37683
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:10 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450330, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:10.902638 26590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:10.865249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.193:43613)
0811 12:12:10.877124 (+ 11875us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:10.877182 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:10.881907 (+ 4725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:10.883390 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:10.883401 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:10.884027 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:10.884756 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:10.884779 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:10.886915 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:10.886920 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:10.888669 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:10.888684 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:10.890399 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:10.890407 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:10.890642 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:10.891370 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:10.891397 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:10.893746 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:10.897714 (+ 3968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:10.897722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:10.897752 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:10.898250 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:10.898758 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:10.898762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:10.898764 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:10.899044 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:10.900116 (+ 1072us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:10.900123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:10.900476 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 12:12:10.900782 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 12:12:10.901281 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11513,"thread_start_us":15007,"threads_started":1}
I20260811 12:12:10.905004 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:10.865250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:43613 (local address 127.4.48.254:42423)
0811 12:12:10.881656 (+ 16406us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:10.881670 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:10.881715 (+ 45us) server_negotiation.cc:407] Connection header received
0811 12:12:10.882530 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:10.882543 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:10.882878 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:10.883204 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:10.885005 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:10.886730 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:10.888853 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:10.889872 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:10.890044 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:10.894029 (+ 3985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:10.894040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:10.894059 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:10.894113 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:10.897565 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:10.898437 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:10.898445 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:10.898452 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:10.898637 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:10.899189 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:10.899194 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:10.899197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:10.899879 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:10.900049 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:10.903354 (+ 3305us) server_negotiation.cc:299] Negotiation successful
0811 12:12:10.903788 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16165,"thread_start_us":15966,"threads_started":1}
I20260811 12:12:10.909373 26586 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:10.909677 26586 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:10.910372 26586 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
W20260811 12:12:10.913434 26586 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:42423 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:12:10.913666 26586 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:10.913939 26586 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:10.915210 26586 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:10.915472 26586 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
W20260811 12:12:10.916605 26586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:12:11.280076 26591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:11.280335 26591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:11.280501 26591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:11.287665 26591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:11.287811 26591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:11.287967 26591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:11.298333 26591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:36477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=37683
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:11.299600 26591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:11.300904 26591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:11.315022 26609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:11.315119 26608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:11.315994 26591 server_base.cc:1034] running on GCE node
W20260811 12:12:11.316257 26616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:11.316932 26591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:11.317879 26591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:11.319096 26591 hybrid_clock.cc:648] HybridClock initialized: now 1786450331319049 us; error 37 us; skew 500 ppm
I20260811 12:12:11.319409 26591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:11 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:11.322778 26591 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:11.348044 26591 webserver.cc:460] Webserver started at http://127.4.48.194:37683/ using document root <none> and password file <none>
I20260811 12:12:11.349140 26591 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:11.349391 26591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:12:11 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:11.387764 26591 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.006s sys 0.001s
I20260811 12:12:11.391520 26631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:11.396207 26591 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260811 12:12:11.396410 26591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "d6df9675b4294503a68aa08310cadb8d"
format_stamp: "Formatted at 2026-08-11 12:12:08 on dist-test-slave-nj21"
server_key: "7a592efdedcee9d2001ffb024a275f9b"
server_key_iv: "26eb1fef90d55d317257a178796b3c01"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:11.397836 26591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:11.427232 26591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:11.432361 26591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:11.432710 26591 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 11 12:12:11 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:11.434376 26591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:11.436205 26591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:11.436363 26591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 12:12:11.436506 26591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:11.436601 26591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:11.521713 26591 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:36477
I20260811 12:12:11.521797 26791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:36477 every 8 connection(s)
I20260811 12:12:11.524022 26591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:11.531010 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 26591
I20260811 12:12:11.531813 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.4.48.212:57035/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:44081
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:42423
--webserver_port=42771
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:11 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:11.551229 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:11.531228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:45837 (local address 127.4.48.254:42423)
0811 12:12:11.531996 (+ 768us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:11.532003 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:11.534734 (+ 2731us) server_negotiation.cc:407] Connection header received
0811 12:12:11.534797 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:11.534801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:11.534867 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:11.534960 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:11.536754 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:11.537611 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:11.539341 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:11.539933 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:11.540028 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:11.543832 (+ 3804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:11.543840 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:11.543845 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:11.543888 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:11.547605 (+ 3717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:11.548554 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:11.548560 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:11.548563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:11.548709 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:11.549233 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:11.549247 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:11.549250 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:11.549610 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:11.549775 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:11.550617 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 12:12:11.550839 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":264,"threads_started":1}
I20260811 12:12:11.552414 26797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:11.530548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.194:45837)
0811 12:12:11.532522 (+ 1974us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:11.532577 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:11.534458 (+ 1881us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:11.535223 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:11.535231 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:11.535657 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:11.536445 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:11.536511 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:11.537768 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:11.537772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:11.539168 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:11.539182 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:11.540311 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:11.540316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:11.540547 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:11.541227 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:11.541254 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:11.543567 (+ 2313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:11.547785 (+ 4218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:11.547794 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:11.547809 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:11.548405 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:11.548823 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:11.548828 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:11.548832 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:11.549094 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:11.549744 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:11.549752 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:11.550159 (+ 407us) client_negotiation.cc:769] Sending connection context
0811 12:12:11.550478 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 12:12:11.551011 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1671,"thread_start_us":1758,"threads_started":1}
I20260811 12:12:11.554004 26792 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:11.554353 26792 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:11.555222 26792 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:11.557365 26276 ts_manager.cc:194] Registered new tserver with Master: d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477)
I20260811 12:12:11.559566 26276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:45837
Aug 11 12:12:11 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:11.916783 26586 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:11.917241 26586 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:11.918551 26276 ts_manager.cc:194] Registered new tserver with Master: 861ed28d44a94372818c4719abcb786e (127.4.48.193:36117)
I20260811 12:12:11.919701 26276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:43613
W20260811 12:12:11.991412 26801 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:11.991852 26801 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:11.992007 26801 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:12.002952 26801 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:12.003113 26801 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:12.003297 26801 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:12.018175 26801 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.4.48.212:57035/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:44081
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=42771
--tserver_master_addrs=127.4.48.254:42423
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:12.019807 26801 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:12.021461 26801 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:12.039220 26809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:12.039589 26811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:12.041242 26801 server_base.cc:1034] running on GCE node
W20260811 12:12:12.045440 26816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:12.046818 26801 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:12.048272 26801 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:12.051729 26801 hybrid_clock.cc:648] HybridClock initialized: now 1786450332051650 us; error 41 us; skew 500 ppm
I20260811 12:12:12.052260 26801 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:12 dist-test-slave-nj21 krb5kdc[24681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450332, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:12.057111 26801 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:12.064391 26801 webserver.cc:460] Webserver started at http://127.4.48.195:42771/ using document root <none> and password file <none>
I20260811 12:12:12.065465 26801 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:12.065620 26801 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 12:12:12 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450332, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
Aug 11 12:12:12 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450331, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for HTTP/127.4.48.212@KRBTEST.COM
I20260811 12:12:12.111423 26801 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.009s sys 0.008s
I20260811 12:12:12.138358 26832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:12.142235 26801 fs_manager.cc:725] Time spent opening block manager: real 0.029s user 0.010s sys 0.016s
I20260811 12:12:12.142549 26801 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "040297a67d5447d58a4ca4a9f0e282c1"
format_stamp: "Formatted at 2026-08-11 12:12:09 on dist-test-slave-nj21"
server_key: "5432bd168a8b981bb1d6d535212fbd6e"
server_key_iv: "c9493a9468b3c8a1fedc57a99d0223ba"
server_key_version: "kuduclusterkey@0"
I20260811 12:12:12.144378 26801 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:12.213287 26801 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:12.218802 26801 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:12.219193 26801 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:12.225694 26801 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:12.228685 26801 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:12.228854 26801 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 12:12:12.229012 26801 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:12.229118 26801 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:12.314652 26801 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:44081
I20260811 12:12:12.314720 26994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:44081 every 8 connection(s)
I20260811 12:12:12.316273 26801 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:12.326717 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 26801
Aug 11 12:12:12 dist-test-slave-nj21 krb5kdc[24681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450332, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:12.344151 26999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.319314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:48931 (local address 127.4.48.254:42423)
0811 12:12:12.324123 (+ 4809us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.324130 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.324162 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:12:12.324838 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.324844 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.325144 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.325355 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.327171 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.328417 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.330877 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.331595 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.331742 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.336208 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.336216 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.336220 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.336261 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.340257 (+ 3996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.341203 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.341208 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.341212 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.341360 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.341927 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.341933 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.341936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.342347 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.342488 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.343474 (+ 986us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.343737 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4675,"thread_start_us":4537,"threads_started":1}
I20260811 12:12:12.345330 27002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.319919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:42423 (local address 127.4.48.195:48931)
0811 12:12:12.322069 (+ 2150us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.322132 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.324535 (+ 2403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.325585 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.325593 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.326121 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:12.326967 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.326995 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.328622 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.328628 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.330662 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.330679 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.332090 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.332098 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.332372 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.333149 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:12.333180 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:12.335936 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:12.340439 (+ 4503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:12.340448 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:12.340464 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:12.341056 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:12.341493 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:12.341498 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:12.341502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:12.341768 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:12.342561 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:12.342570 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:12.343011 (+ 441us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.343314 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.343991 (+ 677us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1816,"thread_start_us":1558,"threads_started":1}
I20260811 12:12:12.347038 26995 heartbeater.cc:347] Connected to a master server at 127.4.48.254:42423
I20260811 12:12:12.347340 26995 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:12.348107 26995 heartbeater.cc:511] Master 127.4.48.254:42423 requested a full tablet report, sending...
I20260811 12:12:12.349644 26276 ts_manager.cc:194] Registered new tserver with Master: 040297a67d5447d58a4ca4a9f0e282c1 (127.4.48.195:44081)
I20260811 12:12:12.350724 26276 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:48931
I20260811 12:12:12.360188 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.327659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58024 (local address 127.4.48.254:42423)
0811 12:12:12.343520 (+ 15861us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.343525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.343541 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:12:12.343595 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.343600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.343665 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.343785 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.346009 (+ 2224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.347334 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.350606 (+ 3272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.351361 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.351480 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.355391 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.355398 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.355403 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.355440 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.358039 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.358639 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.358643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.358645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.358763 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.359055 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.359058 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.359059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.359283 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.359482 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.359736 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.359946 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15710,"thread_start_us":16255,"threads_started":1}
I20260811 12:12:12.362418 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:12:12.377033 27010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.365263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58036 (local address 127.4.48.254:42423)
0811 12:12:12.365415 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.365419 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.365870 (+ 451us) server_negotiation.cc:407] Connection header received
0811 12:12:12.366016 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.366021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.366092 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.366167 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.367536 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.368346 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.369748 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.370164 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.370265 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.372572 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.372578 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.372582 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.372611 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.374983 (+ 2372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.375566 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.375570 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.375572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.375704 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.376000 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.376004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.376007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.376247 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.376423 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.376617 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.376772 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:12:12.381471 26276 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58036:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:12:12.388871 26276 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:12:12.415185 27028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.402137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36477 (local address 127.0.0.1:34974)
0811 12:12:12.402978 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.403040 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.403223 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.408162 (+ 4939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.408170 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.408198 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:12.408621 (+ 423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.408636 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.410788 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.410793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.412718 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.412730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.414209 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.414216 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.414435 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.414499 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.414623 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.414806 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":3,"spinlock_wait_cycles":4736,"thread_start_us":699,"threads_started":1}
I20260811 12:12:12.415980 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.402655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34974 (local address 127.4.48.194:36477)
0811 12:12:12.407490 (+ 4835us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.407499 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.407527 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:12:12.407627 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.407636 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.407928 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.408831 (+ 903us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:12.409023 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.410626 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.412863 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.413934 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.414126 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.414335 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.415448 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.415624 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4681,"thread_start_us":4543,"threads_started":1}
I20260811 12:12:12.416023 27033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.403432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:44081 (local address 127.0.0.1:49138)
0811 12:12:12.404035 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.404054 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.404170 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.408735 (+ 4565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.408739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.408779 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:12.409185 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.409193 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.411211 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.411216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.413424 (+ 2208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.413434 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.415211 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.415217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.415389 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.415419 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.415500 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.415616 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":25,"thread_start_us":983,"threads_started":1}
I20260811 12:12:12.416762 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.404137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49138 (local address 127.4.48.195:44081)
0811 12:12:12.408221 (+ 4084us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.408226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.408254 (+ 28us) server_negotiation.cc:407] Connection header received
0811 12:12:12.408326 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.408331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.408520 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.409345 (+ 825us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:12.409588 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.411077 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.413572 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.414978 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.415107 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.415329 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.416234 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.416415 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3948,"thread_start_us":3795,"threads_started":1}
I20260811 12:12:12.416978 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.401965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39628)
0811 12:12:12.402978 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.403045 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.403223 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.408449 (+ 5226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.408454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.408482 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:12.408997 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.409009 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.410839 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.410848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.413404 (+ 2556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.413420 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.416081 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.416088 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.416377 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.416408 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.416496 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.416634 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":109,"spinlock_wait_cycles":6784,"thread_start_us":687,"threads_started":1}
I20260811 12:12:12.417929 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.402441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39628 (local address 127.4.48.193:36117)
0811 12:12:12.407888 (+ 5447us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.407896 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.407921 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:12:12.408019 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.408027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.408233 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.408449 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:12.409196 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.410710 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.414428 (+ 3718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.415856 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.415986 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.416202 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.416588 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.416746 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5288,"thread_start_us":5100,"threads_started":1}
I20260811 12:12:12.419922 26705 tablet_service.cc:1467] Processing CreateTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad (DEFAULT_TABLE table=test-table [id=de55dd21bfa048a78e7e21f75ab4bf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:12.420231 26901 tablet_service.cc:1467] Processing CreateTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad (DEFAULT_TABLE table=test-table [id=de55dd21bfa048a78e7e21f75ab4bf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:12.421382 26901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df456e35a03b4dbd81ae1bb8e7f25dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:12.421444 26705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df456e35a03b4dbd81ae1bb8e7f25dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:12.421440 26489 tablet_service.cc:1467] Processing CreateTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad (DEFAULT_TABLE table=test-table [id=de55dd21bfa048a78e7e21f75ab4bf55]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:12.422760 26489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df456e35a03b4dbd81ae1bb8e7f25dad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:12.433416 27036 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Bootstrap starting.
I20260811 12:12:12.436412 27037 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Bootstrap starting.
I20260811 12:12:12.436792 27036 tablet_bootstrap.cc:654] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:12.437798 27038 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Bootstrap starting.
I20260811 12:12:12.438119 27036 log.cc:824] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:12.439742 27037 tablet_bootstrap.cc:654] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:12.440989 27037 log.cc:824] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:12.441290 27038 tablet_bootstrap.cc:654] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:12.441701 27036 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: No bootstrap required, opened a new log
I20260811 12:12:12.441955 27036 ts_tablet_manager.cc:1397] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:12:12.442472 27038 log.cc:824] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:12.444020 27037 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: No bootstrap required, opened a new log
I20260811 12:12:12.444271 27037 ts_tablet_manager.cc:1397] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:12:12.445547 27038 tablet_bootstrap.cc:492] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: No bootstrap required, opened a new log
I20260811 12:12:12.445829 27038 ts_tablet_manager.cc:1397] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260811 12:12:12.451956 27036 raft_consensus.cc:348] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.452384 27036 raft_consensus.cc:374] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:12.452486 27036 raft_consensus.cc:729] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 040297a67d5447d58a4ca4a9f0e282c1, State: Initialized, Role: FOLLOWER
I20260811 12:12:12.453073 27036 consensus_queue.cc:260] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.453130 27038 raft_consensus.cc:348] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.453438 27038 raft_consensus.cc:374] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:12.453531 27038 raft_consensus.cc:729] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 861ed28d44a94372818c4719abcb786e, State: Initialized, Role: FOLLOWER
I20260811 12:12:12.454020 27038 consensus_queue.cc:260] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.454290 27037 raft_consensus.cc:348] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.454772 27037 raft_consensus.cc:374] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:12.454905 27037 raft_consensus.cc:729] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6df9675b4294503a68aa08310cadb8d, State: Initialized, Role: FOLLOWER
I20260811 12:12:12.455029 26586 heartbeater.cc:503] Master 127.4.48.254:42423 was elected leader, sending a full tablet report...
I20260811 12:12:12.455641 27038 ts_tablet_manager.cc:1428] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Time spent starting tablet: real 0.010s user 0.007s sys 0.002s
I20260811 12:12:12.455644 27037 consensus_queue.cc:260] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.457021 26792 heartbeater.cc:503] Master 127.4.48.254:42423 was elected leader, sending a full tablet report...
I20260811 12:12:12.457641 27037 ts_tablet_manager.cc:1428] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:12:12.458511 26995 heartbeater.cc:503] Master 127.4.48.254:42423 was elected leader, sending a full tablet report...
I20260811 12:12:12.459107 27036 ts_tablet_manager.cc:1428] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Time spent starting tablet: real 0.017s user 0.012s sys 0.004s
I20260811 12:12:12.496739 27048 raft_consensus.cc:482] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:12.497031 27048 raft_consensus.cc:504] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.498735 27048 leader_election.cc:288] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477), 040297a67d5447d58a4ca4a9f0e282c1 (127.4.48.195:44081)
I20260811 12:12:12.506794 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.500149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:38937 (local address 127.4.48.195:44081)
0811 12:12:12.500356 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.500361 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.500557 (+ 196us) server_negotiation.cc:407] Connection header received
0811 12:12:12.500724 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.500727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.500782 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.500856 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:12.501351 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.502282 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.504560 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.505726 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.505883 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.505955 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.506410 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.506545 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:12:12.506794 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.499058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:48337 (local address 127.4.48.194:36477)
0811 12:12:12.499245 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.499251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.499882 (+ 631us) server_negotiation.cc:407] Connection header received
0811 12:12:12.500099 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.500105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.500194 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.500281 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:12.501012 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.501975 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.504537 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.505726 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.505883 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.505955 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.506410 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.506551 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:12:12.506975 27050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.499080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:36477 (local address 127.4.48.193:48337)
0811 12:12:12.499780 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.499800 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.499931 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.500407 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.500412 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.500439 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:12.500873 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.500885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.502129 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.502134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.504363 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.504380 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.505979 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.505985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.506188 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.506235 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.506332 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.506489 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":28,"spinlock_wait_cycles":21248,"thread_start_us":358,"threads_started":1}
I20260811 12:12:12.507187 27053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.499927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:44081 (local address 127.4.48.193:38937)
0811 12:12:12.500468 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.500485 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:12.500620 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:12.500890 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:12.500894 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:12.500919 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:12.501202 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.501210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.502427 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.502432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.504435 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:12.504446 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.505937 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.505944 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:12.506182 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.506240 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 12:12:12.506332 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 12:12:12.506489 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":79,"thread_start_us":256,"threads_started":1}
I20260811 12:12:12.507958 26738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df456e35a03b4dbd81ae1bb8e7f25dad" candidate_uuid: "861ed28d44a94372818c4719abcb786e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6df9675b4294503a68aa08310cadb8d" is_pre_election: true
I20260811 12:12:12.508153 26926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df456e35a03b4dbd81ae1bb8e7f25dad" candidate_uuid: "861ed28d44a94372818c4719abcb786e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040297a67d5447d58a4ca4a9f0e282c1" is_pre_election: true
I20260811 12:12:12.508524 26738 raft_consensus.cc:2454] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 861ed28d44a94372818c4719abcb786e in term 0.
I20260811 12:12:12.508648 26926 raft_consensus.cc:2454] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 861ed28d44a94372818c4719abcb786e in term 0.
I20260811 12:12:12.509275 26404 leader_election.cc:302] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 040297a67d5447d58a4ca4a9f0e282c1, 861ed28d44a94372818c4719abcb786e; no voters:
I20260811 12:12:12.509619 27048 raft_consensus.cc:2781] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:12:12.509806 27048 raft_consensus.cc:482] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:12:12.509912 27048 raft_consensus.cc:3037] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:12.512234 27048 raft_consensus.cc:504] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.512835 27048 leader_election.cc:288] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [CANDIDATE]: Term 1 election: Requested vote from peers d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477), 040297a67d5447d58a4ca4a9f0e282c1 (127.4.48.195:44081)
I20260811 12:12:12.513259 26738 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df456e35a03b4dbd81ae1bb8e7f25dad" candidate_uuid: "861ed28d44a94372818c4719abcb786e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d6df9675b4294503a68aa08310cadb8d"
I20260811 12:12:12.513320 26926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df456e35a03b4dbd81ae1bb8e7f25dad" candidate_uuid: "861ed28d44a94372818c4719abcb786e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "040297a67d5447d58a4ca4a9f0e282c1"
I20260811 12:12:12.513430 26738 raft_consensus.cc:3037] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:12.513513 26926 raft_consensus.cc:3037] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:12.515559 26738 raft_consensus.cc:2454] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 861ed28d44a94372818c4719abcb786e in term 1.
I20260811 12:12:12.515902 26926 raft_consensus.cc:2454] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 861ed28d44a94372818c4719abcb786e in term 1.
I20260811 12:12:12.515933 26404 leader_election.cc:302] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 861ed28d44a94372818c4719abcb786e, d6df9675b4294503a68aa08310cadb8d; no voters:
I20260811 12:12:12.516201 27048 raft_consensus.cc:2781] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:12.516647 27048 raft_consensus.cc:686] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 1 LEADER]: Becoming Leader. State: Replica: 861ed28d44a94372818c4719abcb786e, State: Running, Role: LEADER
I20260811 12:12:12.517091 27048 consensus_queue.cc:237] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } }
I20260811 12:12:12.521591 26276 catalog_manager.cc:5495] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e reported cstate change: term changed from 0 to 1, leader changed from <none> to 861ed28d44a94372818c4719abcb786e (127.4.48.193). New cstate: current_term: 1 leader_uuid: "861ed28d44a94372818c4719abcb786e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "861ed28d44a94372818c4719abcb786e" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 } health_report { overall_health: UNKNOWN } } }
W20260811 12:12:12.532153 26794 tablet.cc:2369] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:12.563994 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.559066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.4.48.193:36117)
0811 12:12:12.559252 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.559256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.559274 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:12:12.559432 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.559436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.559521 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.559638 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:12:12.560001 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.560709 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.562066 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.562489 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.562576 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.562992 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:12:12.563505 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:12:12.563612 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.563666 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.563748 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 12:12:12.569238 26996 tablet.cc:2369] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:12.571982 26926 raft_consensus.cc:1261] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 1 FOLLOWER]: Refusing update from remote peer 861ed28d44a94372818c4719abcb786e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:12.571975 26738 raft_consensus.cc:1261] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 1 FOLLOWER]: Refusing update from remote peer 861ed28d44a94372818c4719abcb786e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:12.572921 27048 consensus_queue.cc:1035] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [LEADER]: Connected to new peer: Peer: permanent_uuid: "d6df9675b4294503a68aa08310cadb8d" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 36477 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:12.573256 27054 consensus_queue.cc:1035] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [LEADER]: Connected to new peer: Peer: permanent_uuid: "040297a67d5447d58a4ca4a9f0e282c1" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 44081 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:12.600217 26276 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:12.600514 26276 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58036:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:12.604614 26276 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7af9aa743bd34882ad22c831d58e8958: Sending DeleteTablet for 3 replicas of tablet df456e35a03b4dbd81ae1bb8e7f25dad
I20260811 12:12:12.605772 26705 tablet_service.cc:1514] Processing DeleteTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:12 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:34974
I20260811 12:12:12.606014 26489 tablet_service.cc:1514] Processing DeleteTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:12 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:39628
I20260811 12:12:12.606009 26901 tablet_service.cc:1514] Processing DeleteTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:12 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:49138
I20260811 12:12:12.606596 26704 tablet_service.cc:1514] Processing DeleteTablet for tablet df456e35a03b4dbd81ae1bb8e7f25dad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:12 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:34974
I20260811 12:12:12.615366 27075 tablet_replica.cc:330] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: stopping tablet replica
I20260811 12:12:12.615644 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 26332
I20260811 12:12:12.615944 27073 tablet_replica.cc:330] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: stopping tablet replica
I20260811 12:12:12.616719 27075 raft_consensus.cc:2229] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:12:12.617460 27075 raft_consensus.cc:2258] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:12.618377 27076 tablet_replica.cc:330] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: stopping tablet replica
I20260811 12:12:12.618757 27076 raft_consensus.cc:2229] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:12.619239 27076 raft_consensus.cc:2258] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:12.619477 27073 raft_consensus.cc:2229] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:12.619936 27073 raft_consensus.cc:2258] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:12.621490 27076 ts_tablet_manager.cc:1905] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:12.623741 27075 ts_tablet_manager.cc:1905] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:12.628084 27076 ts_tablet_manager.cc:1918] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:12.628261 27076 log.cc:1196] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/df456e35a03b4dbd81ae1bb8e7f25dad
I20260811 12:12:12.628784 27076 ts_tablet_manager.cc:1939] T df456e35a03b4dbd81ae1bb8e7f25dad P 040297a67d5447d58a4ca4a9f0e282c1: Deleting consensus metadata
I20260811 12:12:12.629326 27073 ts_tablet_manager.cc:1905] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:12.630560 26249 catalog_manager.cc:4841] TS 040297a67d5447d58a4ca4a9f0e282c1 (127.4.48.195:44081): tablet df456e35a03b4dbd81ae1bb8e7f25dad (table test-table [id=de55dd21bfa048a78e7e21f75ab4bf55]) successfully deleted
I20260811 12:12:12.634580 27029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.616867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.4.48.193:36117)
0811 12:12:12.617014 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.617019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.617042 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:12:12.617249 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.617253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.617342 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.617450 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.619132 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.620337 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.622010 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.622690 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.622840 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.626253 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.626267 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.626284 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.626337 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.630669 (+ 4332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.631696 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.631705 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.631709 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.631884 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.632310 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.632315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.632318 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.633247 (+ 929us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.633543 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.633861 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.634180 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:12:12.635243 27073 ts_tablet_manager.cc:1918] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:12.635293 26579 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:12.635404 27073 log.cc:1196] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/df456e35a03b4dbd81ae1bb8e7f25dad
I20260811 12:12:12.636258 27073 ts_tablet_manager.cc:1939] T df456e35a03b4dbd81ae1bb8e7f25dad P d6df9675b4294503a68aa08310cadb8d: Deleting consensus metadata
I20260811 12:12:12.638007 26249 catalog_manager.cc:4841] TS d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477): tablet df456e35a03b4dbd81ae1bb8e7f25dad (table test-table [id=de55dd21bfa048a78e7e21f75ab4bf55]) successfully deleted
W20260811 12:12:12.638563 26249 catalog_manager.cc:4808] TS d6df9675b4294503a68aa08310cadb8d (127.4.48.194:36477): delete failed for tablet df456e35a03b4dbd81ae1bb8e7f25dad because the tablet was not found. No further retry: Not found: Tablet not found: df456e35a03b4dbd81ae1bb8e7f25dad
I20260811 12:12:12.731521 27075 ts_tablet_manager.cc:1918] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:12.731873 27075 log.cc:1196] T df456e35a03b4dbd81ae1bb8e7f25dad P 861ed28d44a94372818c4719abcb786e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/df456e35a03b4dbd81ae1bb8e7f25dad
I20260811 12:12:12.732300 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 26332
W20260811 12:12:12.758519 26261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.48.193:36117 (error 108)
I20260811 12:12:12.758709 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 26591
I20260811 12:12:12.760334 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.759895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39652)
0811 12:12:12.760053 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.760161 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:12:12.760557 26261 catalog_manager.cc:4584] TS 861ed28d44a94372818c4719abcb786e (127.4.48.193:36117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df456e35a03b4dbd81ae1bb8e7f25dad: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
I20260811 12:12:12.772022 27031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.759513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.4.48.194:36477)
0811 12:12:12.759691 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.759697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.759766 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:12:12.759955 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.759960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.760063 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.760152 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.761557 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.762268 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.763436 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.763924 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.764017 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.766736 (+ 2719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.766745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.766760 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.766784 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.769387 (+ 2603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.770114 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.770121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.770123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.770247 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.770555 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.770557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.770559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.770904 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.771125 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.771492 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.771658 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:12:12.772753 26787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:12.814260 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.813731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39654)
0811 12:12:12.813968 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.814085 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 12:12:12.815377 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.815002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39662)
0811 12:12:12.815151 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.815249 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:12:12.859906 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 26591
I20260811 12:12:12.875996 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.875408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39666)
0811 12:12:12.875649 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.875801 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 12:12:12.877030 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.876665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39682)
0811 12:12:12.876806 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.876888 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 12:12:12.884498 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 26801
I20260811 12:12:12.896874 27035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.885173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.4.48.195:44081)
0811 12:12:12.885348 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:12.885352 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:12.885370 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:12:12.885473 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:12.885476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:12.885542 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:12.885620 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:12.886835 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.887613 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.888974 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:12.889355 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:12.889455 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:12.891966 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:12.891982 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:12.892000 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:12.892036 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:12.894478 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.895087 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.895093 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.895095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.895213 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:12.895485 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:12.895488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:12.895490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:12.895898 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:12.896124 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:12.896378 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 12:12:12.896539 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:12:12.897526 26985 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:12.957602 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.957014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39688)
0811 12:12:12.957269 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.957392 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 12:12:12.958752 27027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:12.958308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36117 (local address 127.0.0.1:39696)
0811 12:12:12.958510 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:12.958602 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 12:12:12.985194 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 26801
I20260811 12:12:13.008373 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 26224
I20260811 12:12:13.009562 26320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:13.118474 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 26224
I20260811 12:12:13.148900 4291 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 12:12:13.493405 4291 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 12:12:13.493623 4291 mini_ranger.cc:66] Stopping Ranger...
I20260811 12:12:13.837598 4291 mini_ranger.cc:68] Stopped Ranger
2026-08-11 12:12:13.837 UTC [24731] LOG: received smart shutdown request
2026-08-11 12:12:13.840 UTC [24731] LOG: background worker "logical replication launcher" (PID 24745) exited with exit code 1
2026-08-11 12:12:13.840 UTC [24738] LOG: shutting down
2026-08-11 12:12:13.976 UTC [24731] LOG: database system is shut down
2026-08-11T12:12:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49977 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:12:14 dist-test-slave-nj21 krb5kdc[27094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:12:14 dist-test-slave-nj21 krb5kdc[27094](info): set up 2 sockets
Aug 11 12:12:14 dist-test-slave-nj21 krb5kdc[27094](info): commencing operation
krb5kdc: starting...
W20260811 12:12:15.311292 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:12:15 dist-test-slave-nj21 krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:12:15.553026 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46457
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:15.879552 27118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:15.879844 27118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:15.879938 27118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:15.886842 27118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:15.886936 27118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:15.886988 27118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:15.887043 27118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:15.887091 27118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:15.896347 27118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:15.897511 27118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:15.898710 27118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:15.909780 27124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:15.909684 27125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:15.909724 27127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:15.910929 27118 server_base.cc:1034] running on GCE node
I20260811 12:12:15.911645 27118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:15.912971 27118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:15.914175 27118 hybrid_clock.cc:648] HybridClock initialized: now 1786450335914124 us; error 44 us; skew 500 ppm
I20260811 12:12:15.914511 27118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:15 dist-test-slave-nj21 krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:15.917513 27118 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:15.918813 27118 webserver.cc:460] Webserver started at http://127.4.48.254:46353/ using document root <none> and password file <none>
I20260811 12:12:15.919421 27118 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:15.919515 27118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:15.919807 27118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:15.922644 27118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "3754b467081c4054beedb10412faffaa"
format_stamp: "Formatted at 2026-08-11 12:12:15 on dist-test-slave-nj21"
server_key: "1b92100ede8b2de31aa7f1e293a5b898"
server_key_iv: "97f08cea1543c18b4933213e20ddc794"
server_key_version: "encryptionkey@0"
I20260811 12:12:15.923430 27118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "3754b467081c4054beedb10412faffaa"
format_stamp: "Formatted at 2026-08-11 12:12:15 on dist-test-slave-nj21"
server_key: "1b92100ede8b2de31aa7f1e293a5b898"
server_key_iv: "97f08cea1543c18b4933213e20ddc794"
server_key_version: "encryptionkey@0"
I20260811 12:12:15.928182 27118 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:12:15.931362 27144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:15.932638 27118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:15.932778 27118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "3754b467081c4054beedb10412faffaa"
format_stamp: "Formatted at 2026-08-11 12:12:15 on dist-test-slave-nj21"
server_key: "1b92100ede8b2de31aa7f1e293a5b898"
server_key_iv: "97f08cea1543c18b4933213e20ddc794"
server_key_version: "encryptionkey@0"
I20260811 12:12:15.932930 27118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:15.950762 27118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:15.954697 27118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:15.954924 27118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:15.972031 27118 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:46457
I20260811 12:12:15.972077 27223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:46457 every 8 connection(s)
I20260811 12:12:15.973356 27118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:15.976984 27225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:15.983034 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27118
I20260811 12:12:15.983232 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:12:15.983892 4291 external_mini_cluster.cc:1428] Setting key 31b83a24f4a107c9308ddbc8b98f92b2
I20260811 12:12:15.983263 27225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:12:15 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:16.004340 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:15.986150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46457 (local address 127.0.0.1:39382)
0811 12:12:15.986975 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:15.987021 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:15.988920 (+ 1899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:15.989697 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:15.989703 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:15.990360 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:15.990696 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:15.990713 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:15.992247 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:15.992251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:15.993297 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:15.993309 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:15.994201 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:15.994208 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:15.994373 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:15.994990 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:15.995012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:15.997218 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:16.000783 (+ 3565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.000793 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.000801 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.001181 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.001607 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.001610 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.001612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.001755 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.002570 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:16.002576 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:16.002754 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 12:12:16.002960 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 12:12:16.003300 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":357,"threads_started":1}
I20260811 12:12:16.004388 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:15.986150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.4.48.254:46457)
0811 12:12:15.987764 (+ 1614us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:15.987772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:15.987821 (+ 49us) server_negotiation.cc:407] Connection header received
0811 12:12:15.989248 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:15.989254 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:15.989481 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:15.989648 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:15.990885 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:15.992107 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:15.993435 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:15.993981 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:15.994139 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:15.997459 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:15.997469 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:15.997483 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:15.997517 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.000661 (+ 3144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.001310 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.001318 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.001321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.001502 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.001857 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.001861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.001864 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.002448 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.002667 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.003116 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.003458 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":1381,"thread_start_us":1185,"threads_started":1}
I20260811 12:12:16.006963 27225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:12:16.007987 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.007485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53598)
0811 12:12:16.007700 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.007828 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 12:12:16.008224 27147 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
I20260811 12:12:16.009732 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.009326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53604)
0811 12:12:16.009502 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.009588 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
WARNING: no policy specified for kudu/127.4.48.253@KRBTEST.COM; defaulting to no policy
W20260811 12:12:16.009941 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:45603: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
I20260811 12:12:16.040395 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:45603
--webserver_interface=127.4.48.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:16.059334 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } attempt: 1
I20260811 12:12:16.060544 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.060106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53618)
0811 12:12:16.060302 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.060401 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:12:16.061503 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.061190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53628)
0811 12:12:16.061305 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.061381 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:12:16.061691 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:45603: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
I20260811 12:12:16.129920 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } attempt: 2
I20260811 12:12:16.131095 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.130566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53634)
0811 12:12:16.130838 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.130934 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 12:12:16.132201 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.131857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53644)
0811 12:12:16.131987 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.132070 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:12:16.132378 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:45603: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
I20260811 12:12:16.223596 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } attempt: 3
I20260811 12:12:16.224707 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.224297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53656)
0811 12:12:16.224481 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.224573 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 12:12:16.225613 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.225305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53666)
0811 12:12:16.225414 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.225489 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 12:12:16.225806 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:45603: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
I20260811 12:12:16.369011 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } attempt: 4
I20260811 12:12:16.370194 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.369685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53680)
0811 12:12:16.369950 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.370052 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148}
I20260811 12:12:16.371104 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.370787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53686)
0811 12:12:16.370903 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.370988 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:12:16.371287 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:45603: Network error: Client connection negotiation failed: client connection to 127.4.48.253:45603: connect: Connection refused (error 111)
W20260811 12:12:16.374979 27237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:16.375226 27237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:16.375352 27237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:16.382316 27237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:16.382412 27237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:16.382480 27237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:16.382552 27237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:16.382607 27237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:16.392419 27237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:45603
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.48.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:16.393690 27237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:16.394975 27237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:16.406401 27243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:16.406365 27247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:16.406438 27244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:16.407549 27237 server_base.cc:1034] running on GCE node
I20260811 12:12:16.408315 27237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:16.409608 27237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:16.410835 27237 hybrid_clock.cc:648] HybridClock initialized: now 1786450336410775 us; error 45 us; skew 500 ppm
I20260811 12:12:16.411154 27237 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:16.414115 27237 init.cc:377] Logged in from keytab as kudu/127.4.48.253@KRBTEST.COM (short username kudu)
I20260811 12:12:16.415498 27237 webserver.cc:460] Webserver started at http://127.4.48.253:33871/ using document root <none> and password file <none>
I20260811 12:12:16.416159 27237 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:16.416257 27237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:16.416504 27237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:16.419282 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data/instance:
uuid: "f919d2ffac874fddb5fc6c701928c431"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "66cd95e532e65523506ca4be61178ca5"
server_key_iv: "ae420904e764704572ded67bccf943af"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.420075 27237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal/instance:
uuid: "f919d2ffac874fddb5fc6c701928c431"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "66cd95e532e65523506ca4be61178ca5"
server_key_iv: "ae420904e764704572ded67bccf943af"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.424952 27237 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:12:16.428310 27258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:16.429594 27237 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:12:16.429768 27237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "f919d2ffac874fddb5fc6c701928c431"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "66cd95e532e65523506ca4be61178ca5"
server_key_iv: "ae420904e764704572ded67bccf943af"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.429941 27237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:16.445801 27237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:16.449333 27237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:16.449568 27237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:16.466035 27237 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.253:45603
I20260811 12:12:16.466073 27341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.253:45603 every 8 connection(s)
I20260811 12:12:16.467247 27237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
I20260811 12:12:16.470645 27344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:16.473570 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27237
I20260811 12:12:16.473766 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-1/wal/instance
I20260811 12:12:16.474354 4291 external_mini_cluster.cc:1428] Setting key 4ce7bfcf18cc7f097a468e944b3da68f
I20260811 12:12:16.477687 27344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:16.496598 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.480012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.4.48.254:46457)
0811 12:12:16.480211 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.480218 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.481375 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 12:12:16.483550 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.483554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.483619 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.483738 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.485316 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.485985 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.487458 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.487924 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.488037 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.491179 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:16.491184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:16.491187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:16.491220 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.493716 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.494373 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.494379 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.494381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.494500 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.494897 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.494901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.494903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.495172 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.495300 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.496203 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.496383 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:12:16.497881 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.480449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46457 (local address 127.0.0.1:39384)
0811 12:12:16.481197 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.481251 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:16.483279 (+ 2028us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:16.483944 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:16.483954 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:16.484598 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:16.485145 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.485162 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.486137 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.486140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.487309 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.487321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.488138 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.488145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.488357 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.488957 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:16.488979 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:16.490978 (+ 1999us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:16.493852 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.493861 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.493875 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.494263 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.494609 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.494614 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.494618 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.494788 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.495313 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:16.495321 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:16.495777 (+ 456us) client_negotiation.cc:769] Sending connection context
0811 12:12:16.496106 (+ 329us) client_negotiation.cc:240] Negotiation successful
0811 12:12:16.496599 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":301,"threads_started":1}
WARNING: no policy specified for kudu/127.4.48.252@KRBTEST.COM; defaulting to no policy
I20260811 12:12:16.499444 27344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.252@KRBTEST.COM" created.
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:12:16.512915 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.500236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53694)
0811 12:12:16.500427 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.500450 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:16.500546 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:16.501324 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:16.501337 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:16.501695 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:16.501852 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.501861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.503164 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.503169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.504360 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.504374 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.505120 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.505126 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.505292 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.505787 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:16.505808 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:16.508065 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:16.510807 (+ 2742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.510812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.510816 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.511140 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.511468 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.511472 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.511476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.511608 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.512258 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:16.512265 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:16.512398 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 12:12:16.512511 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:12:16.512686 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 12:12:16.513428 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.500368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53694 (local address 127.4.48.253:45603)
0811 12:12:16.500975 (+ 607us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.500981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.501002 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:12:16.501041 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.501046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.501191 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.501299 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.502010 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.503026 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.504485 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.504980 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.505068 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.508255 (+ 3187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:16.508265 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:16.508277 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:16.508304 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.510693 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.511252 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.511258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.511260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.511371 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.511728 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.511732 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.511735 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.512123 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.512311 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.512677 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.512918 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":353,"threads_started":1}
I20260811 12:12:16.515722 27344 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:12:16.516660 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.516275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39436)
0811 12:12:16.516426 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.516537 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 12:12:16.516894 27261 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.518314 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.518029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39446)
0811 12:12:16.518152 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.518224 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:12:16.518522 27344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
I20260811 12:12:16.532225 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:36039
--webserver_interface=127.4.48.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:16.567783 27344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 1
I20260811 12:12:16.568866 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.568449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39448)
0811 12:12:16.568623 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.568721 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:12:16.569821 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.569494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39450)
0811 12:12:16.569640 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.569718 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:12:16.570016 27344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.638201 27344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 2
I20260811 12:12:16.639211 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.638783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39466)
0811 12:12:16.638972 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.639059 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:12:16.640239 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.639908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39480)
0811 12:12:16.640061 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.640127 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:12:16.640442 27344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.670513 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } attempt: 5
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:12:16.684717 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.671103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53700)
0811 12:12:16.671332 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.671355 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:16.671494 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:16.671876 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:16.671882 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:16.672454 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:16.672716 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.672730 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.673684 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.673689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.675101 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.675118 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.675871 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.675877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.676051 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.676636 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:16.676659 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:16.678969 (+ 2310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:16.682614 (+ 3645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.682618 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.682621 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.682971 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.683330 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.683333 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.683335 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.683452 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.684040 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:16.684044 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:16.684156 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 12:12:16.684293 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:12:16.684481 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:12:16.684985 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.671250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53700 (local address 127.4.48.253:45603)
0811 12:12:16.671389 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.671393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.671414 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:12:16.671607 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.671611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.671727 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.671808 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.672863 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.673526 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.675241 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.675731 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.675823 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.679166 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:16.679172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:16.679177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:16.679218 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.682476 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.683079 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.683086 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.683089 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.683237 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.683541 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.683546 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.683549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.683937 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.684072 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.684437 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.684661 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 12:12:16.685961 27225 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:12:16.686964 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.686502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39496)
0811 12:12:16.686684 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.686785 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:12:16.687983 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.687549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39512)
0811 12:12:16.687744 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.687834 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 12:12:16.688197 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.731657 27344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 3
I20260811 12:12:16.732748 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.732282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39526)
0811 12:12:16.732496 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.732594 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:12:16.733940 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.733423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39536)
0811 12:12:16.733674 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.733778 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
W20260811 12:12:16.734194 27344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.739385 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 1
I20260811 12:12:16.740420 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.739893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39552)
0811 12:12:16.740093 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.740201 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:12:16.741405 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.740985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39554)
0811 12:12:16.741147 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.741257 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:12:16.741617 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.809820 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 2
I20260811 12:12:16.810968 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.810382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39560)
0811 12:12:16.810634 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.810767 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 12:12:16.812036 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.811608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39576)
0811 12:12:16.811794 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.811886 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:12:16.812261 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
W20260811 12:12:16.858984 27356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:16.859241 27356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:16.859335 27356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:16.866168 27356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:16.866256 27356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:16.866307 27356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:16.866362 27356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:16.866405 27356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:16.875638 27356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46457,127.4.48.253:45603,127.4.48.252:36039
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:36039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.48.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:16.876843 27356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:16.877440 27344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 4
I20260811 12:12:16.878237 27356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 12:12:16.878610 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.878056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39578)
0811 12:12:16.878308 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.878423 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:12:16.879644 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.879279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39590)
0811 12:12:16.879434 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.879517 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:12:16.879914 27344 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
W20260811 12:12:16.889565 27365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:16.889541 27364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:16.889670 27367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:16.890261 27356 server_base.cc:1034] running on GCE node
I20260811 12:12:16.890936 27356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:16.892192 27356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:16.893388 27356 hybrid_clock.cc:648] HybridClock initialized: now 1786450336893346 us; error 31 us; skew 500 ppm
I20260811 12:12:16.893699 27356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:16.896562 27356 init.cc:377] Logged in from keytab as kudu/127.4.48.252@KRBTEST.COM (short username kudu)
I20260811 12:12:16.897938 27356 webserver.cc:460] Webserver started at http://127.4.48.252:37441/ using document root <none> and password file <none>
I20260811 12:12:16.898554 27356 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:16.898661 27356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:16.898953 27356 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:16.901755 27356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data/instance:
uuid: "96bb3219f84444bea2afa6b32512148f"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "3b89509c8d3c90182e5c42d04c6e0d65"
server_key_iv: "4a7f7fa99e8a65ba4400d0fe2035b0d0"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.902542 27356 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal/instance:
uuid: "96bb3219f84444bea2afa6b32512148f"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "3b89509c8d3c90182e5c42d04c6e0d65"
server_key_iv: "4a7f7fa99e8a65ba4400d0fe2035b0d0"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.907287 27356 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 12:12:16.910456 27378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:16.911876 27356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:16.912050 27356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "96bb3219f84444bea2afa6b32512148f"
format_stamp: "Formatted at 2026-08-11 12:12:16 on dist-test-slave-nj21"
server_key: "3b89509c8d3c90182e5c42d04c6e0d65"
server_key_iv: "4a7f7fa99e8a65ba4400d0fe2035b0d0"
server_key_version: "encryptionkey@0"
I20260811 12:12:16.912235 27356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:16.918493 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 3
I20260811 12:12:16.919556 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.919071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39600)
0811 12:12:16.919268 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.919368 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 12:12:16.920603 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.920211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39602)
0811 12:12:16.920378 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.920476 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:12:16.920825 27225 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:36039: Network error: Client connection negotiation failed: client connection to 127.4.48.252:36039: connect: Connection refused (error 111)
I20260811 12:12:16.937496 27356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:16.941452 27356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:16.941695 27356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:16.959652 27356 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.252:36039
I20260811 12:12:16.959712 27451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.252:36039 every 8 connection(s)
I20260811 12:12:16.960975 27356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
I20260811 12:12:16.964177 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27356
I20260811 12:12:16.964375 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786450132843697-4291-0/minicluster-data/master-2/wal/instance
I20260811 12:12:16.964798 27452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:16.964927 4291 external_mini_cluster.cc:1428] Setting key 11a37ab6a716ba32047668fa6644274f
I20260811 12:12:16.973716 27452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:16.989385 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.966650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39618 (local address 127.4.48.252:36039)
0811 12:12:16.967414 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.967424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.967466 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:12:16.969325 (+ 1859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.969346 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.969715 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.970021 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.971840 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.973885 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.975900 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.976941 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.977140 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.982276 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:16.982288 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:16.982303 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:16.982344 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.985301 (+ 2957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.986126 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.986133 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.986138 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.986277 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.986665 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.986668 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.986670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.987138 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.987336 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.987889 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.988298 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":339,"threads_started":1}
I20260811 12:12:16.990259 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.976399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.4.48.254:46457)
0811 12:12:16.976570 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.976575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.977346 (+ 771us) server_negotiation.cc:407] Connection header received
0811 12:12:16.977586 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.977591 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.977651 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.977731 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.978854 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.979556 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.980927 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.981389 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.981484 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.984997 (+ 3513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:16.985002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:16.985005 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:16.985036 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:16.987707 (+ 2671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.988515 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.988519 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.988522 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.988650 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:16.989004 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:16.989007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:16.989009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:16.989234 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:16.989347 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:16.989866 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 12:12:16.990027 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:12:16.990459 27459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.976431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:46457 (local address 127.0.0.1:39392)
0811 12:12:16.977207 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.977254 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:16.977460 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:16.977800 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:16.977806 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:16.978463 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:16.978714 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.978723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.979719 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.979724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.980755 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.980770 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.981620 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.981628 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.981804 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.982424 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:16.982444 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:16.984819 (+ 2375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:16.987845 (+ 3026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.987855 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.987861 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.988410 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.988754 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:16.988758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:16.988761 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:16.988909 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:16.989363 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:16.989374 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:16.989581 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 12:12:16.989772 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 12:12:16.990057 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":399,"threads_started":1}
I20260811 12:12:16.991835 27452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
Aug 11 12:12:16 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:17.005323 27459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.992526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:45603 (local address 127.0.0.1:53704)
0811 12:12:16.992746 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:16.992763 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:16.992865 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:16.993192 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:16.993197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:16.993543 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:16.993666 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.993673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.994867 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.994873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.996050 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:16.996061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.996918 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.996922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:16.997069 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:16.997542 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:16.997564 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:16.999842 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:17.003113 (+ 3271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.003117 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.003119 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.003462 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.003869 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.003872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.003874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.004008 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.004483 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:17.004488 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:17.004623 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 12:12:17.004741 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 12:12:17.004997 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122}
I20260811 12:12:17.005424 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.992518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53704 (local address 127.4.48.253:45603)
0811 12:12:16.992697 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.992703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.992841 (+ 138us) server_negotiation.cc:407] Connection header received
0811 12:12:16.992992 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.992997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.993078 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.993162 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.993819 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.994742 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.996180 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.996792 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.996929 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.000017 (+ 3088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.000022 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.000025 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.000057 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.002994 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.003575 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.003580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.003583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.003748 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.004110 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.004116 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.004119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.004373 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.004518 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.004849 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.005050 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:12:17.006304 27452 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:12:17.007293 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:16.992681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.4.48.254:46457)
0811 12:12:16.992840 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:16.992855 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:16.992940 (+ 85us) server_negotiation.cc:407] Connection header received
0811 12:12:16.993087 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:16.993090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:16.993143 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:16.993229 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:16.994549 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.995552 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.997126 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:16.997668 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:16.997791 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.002143 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.002149 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.002153 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.002189 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.004733 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.005351 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.005357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.005361 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.005526 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.005946 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.005950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.005952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.006282 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.006560 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.006802 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.006993 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:12:17 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
Aug 11 12:12:17 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:12:17.018607 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.007207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39620 (local address 127.4.48.252:36039)
0811 12:12:17.007346 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.007352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.007369 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:12:17.007414 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.007419 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.007484 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.007566 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.008291 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.009096 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.010152 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.010545 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.010644 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.013425 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.013431 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.013436 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.013470 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.016301 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.016939 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.016944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.016947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.017103 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.017468 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.017471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.017473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.017725 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.017866 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.018212 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.018390 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:12:17.019071 27459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.007006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39620)
0811 12:12:17.007156 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:17.007173 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:17.007259 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:17.007614 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:17.007619 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:17.008009 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:17.008142 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.008153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.009204 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.009210 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.010030 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.010040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.010667 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.010673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.010843 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.011282 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:17.011297 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:17.013252 (+ 1955us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:17.016424 (+ 3172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.016427 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.016430 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.016814 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.017218 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.017222 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.017226 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.017371 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.017855 (+ 484us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:17.017859 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:17.018001 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 12:12:17.018146 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 12:12:17.018581 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":120}
I20260811 12:12:17.022614 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.008846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.4.48.253:45603)
0811 12:12:17.008988 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.008993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.009027 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:12:17.009175 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.009180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.009239 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.009313 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.010469 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.011141 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.012367 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.012805 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.012899 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.017090 (+ 4191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.017094 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.017097 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.017125 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.020225 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.020823 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.020829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.020832 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.020965 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.021311 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.021316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.021320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.021570 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.021804 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.022131 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.022340 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:12:17.023989 27452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: Bootstrap starting.
I20260811 12:12:17.026955 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:12:17.027480 27452 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:17.028501 27452 log.cc:824] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:17.031165 27452 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: No bootstrap required, opened a new log
I20260811 12:12:17.042867 27452 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.043279 27452 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:17.043398 27452 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96bb3219f84444bea2afa6b32512148f, State: Initialized, Role: FOLLOWER
I20260811 12:12:17.044206 27452 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.050451 27231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.031418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.4.48.254:46457)
0811 12:12:17.031586 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.031592 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.032385 (+ 793us) server_negotiation.cc:407] Connection header received
0811 12:12:17.032637 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.032643 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.032725 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.032792 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.035348 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.036575 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.038922 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.039544 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.039642 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.043328 (+ 3686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.043335 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.043340 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.043380 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.047124 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.048013 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.048019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.048022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.048188 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.048632 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.048638 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.048641 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.049028 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.049244 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.049855 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.050103 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:12:17.051203 27470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.051607 27470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.053475 27452 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:17.055058 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.031519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53730 (local address 127.4.48.253:45603)
0811 12:12:17.031752 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.031757 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.038255 (+ 6498us) server_negotiation.cc:407] Connection header received
0811 12:12:17.038690 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.038696 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.038768 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.038836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.041188 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.042217 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.044201 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.044822 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.044914 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.048210 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.048217 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.048221 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.048257 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.051403 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.053124 (+ 1721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.053130 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.053133 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.053258 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.053670 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.053675 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.053679 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.053967 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.054186 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.054564 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.054778 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:12:17.061040 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.032647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.4.48.252:36039)
0811 12:12:17.035988 (+ 3341us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.035994 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.044331 (+ 8337us) server_negotiation.cc:407] Connection header received
0811 12:12:17.044845 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.044852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.044932 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.044998 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.047118 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.048263 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.050308 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.051065 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.051162 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.054862 (+ 3700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.054869 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.054875 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.054916 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.058258 (+ 3342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.058881 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.058886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.058890 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.059048 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.059343 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.059347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.059351 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.059792 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.060083 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.060620 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.060778 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3211}
W20260811 12:12:17.069350 27484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:12:17.070712 27484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:12:17.066892 27356 master_runner.cc:186] Error getting master registration for 127.4.48.254:46457: OK, instance_id { permanent_uuid: "3754b467081c4054beedb10412faffaa" instance_seqno: 1786450335967105 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:12:17.077992 27356 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:12:17.098022 27225 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 4
Aug 11 12:12:17 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450335, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:12:17.111606 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.098595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39646 (local address 127.4.48.252:36039)
0811 12:12:17.098754 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.098760 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.098829 (+ 69us) server_negotiation.cc:407] Connection header received
0811 12:12:17.098992 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.098997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.099056 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.099138 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.100128 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.101192 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.102395 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.102874 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.102995 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.106343 (+ 3348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.106349 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.106352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.106401 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.109005 (+ 2604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.109751 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.109755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.109757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.109883 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.110275 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.110278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.110280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.110566 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.110669 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.111160 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.111328 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:12:17.111718 27229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.098557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39646)
0811 12:12:17.098745 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:17.098765 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:17.098879 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:17.099182 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:17.099187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:17.099745 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:17.099963 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.099976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.101332 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.101338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.102250 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.102263 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.103022 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.103027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.103192 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.103757 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:17.103774 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:17.106163 (+ 2389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:17.109133 (+ 2970us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.109139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.109142 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.109648 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.109986 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.109991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.109995 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.110176 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.110677 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:17.110682 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:17.110866 (+ 184us) client_negotiation.cc:769] Sending connection context
0811 12:12:17.111029 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 12:12:17.111331 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:12:17.116541 27225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: Bootstrap starting.
I20260811 12:12:17.119588 27225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:17.120636 27225 log.cc:824] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:17.123080 27225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: No bootstrap required, opened a new log
I20260811 12:12:17.130800 27225 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.131085 27225 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:17.131196 27225 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3754b467081c4054beedb10412faffaa, State: Initialized, Role: FOLLOWER
I20260811 12:12:17.131758 27225 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.132607 27489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.133003 27489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.133821 27225 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:17.141921 27470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:17.142338 27470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.143893 27470 leader_election.cc:288] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3754b467081c4054beedb10412faffaa (127.4.48.254:46457), f919d2ffac874fddb5fc6c701928c431 (127.4.48.253:45603)
I20260811 12:12:17.145681 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96bb3219f84444bea2afa6b32512148f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3754b467081c4054beedb10412faffaa" is_pre_election: true
I20260811 12:12:17.146303 27185 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 96bb3219f84444bea2afa6b32512148f in term 0.
I20260811 12:12:17.147094 27384 leader_election.cc:302] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3754b467081c4054beedb10412faffaa, 96bb3219f84444bea2afa6b32512148f; no voters:
I20260811 12:12:17.147619 27470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:12:17.147779 27470 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:12:17.147886 27470 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:17.151147 27470 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.152329 27185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96bb3219f84444bea2afa6b32512148f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3754b467081c4054beedb10412faffaa"
I20260811 12:12:17.152529 27185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:17.143589 27118 master_runner.cc:186] Error getting master registration for 127.4.48.253:45603: OK, instance_id { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" instance_seqno: 1786450336461416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:12:17.155026 27118 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:12:17.156706 27185 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 96bb3219f84444bea2afa6b32512148f in term 1.
W20260811 12:12:17.157860 27503 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:12:17.157979 27503 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:12:17.158628 27470 leader_election.cc:288] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [CANDIDATE]: Term 1 election: Requested vote from peers 3754b467081c4054beedb10412faffaa (127.4.48.254:46457), f919d2ffac874fddb5fc6c701928c431 (127.4.48.253:45603)
I20260811 12:12:17.159262 27384 leader_election.cc:302] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3754b467081c4054beedb10412faffaa, 96bb3219f84444bea2afa6b32512148f; no voters:
I20260811 12:12:17.159597 27470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:17.171919 27470 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [term 1 LEADER]: Becoming Leader. State: Replica: 96bb3219f84444bea2afa6b32512148f, State: Running, Role: LEADER
I20260811 12:12:17.172501 27470 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.179132 27344 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } attempt: 5
I20260811 12:12:17.184785 27505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bb3219f84444bea2afa6b32512148f. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.185199 27505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: This master's current role is: LEADER
Aug 11 12:12:17 dist-test-slave-nj21 krb5kdc[27094](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450336, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
W20260811 12:12:17.193620 27387 tablet.cc:2369] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:17.199923 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.179710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:36039 (local address 127.0.0.1:39656)
0811 12:12:17.180047 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:17.180066 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:17.180167 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:17.180974 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:17.180979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:17.181508 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:17.181714 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.181726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.183225 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.183231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.184604 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:17.184619 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.186187 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.186194 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:17.186400 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.187087 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:17.187112 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:17.189954 (+ 2842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:17.196205 (+ 6251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.196210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.196233 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.196763 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.197240 (+ 477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:17.197245 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:17.197249 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:17.197419 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:17.198264 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:17.198270 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:17.198525 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 12:12:17.198745 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 12:12:17.199176 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260811 12:12:17.201493 27457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:17.180355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39656 (local address 127.4.48.252:36039)
0811 12:12:17.180509 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:17.180514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:17.180530 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:12:17.180579 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:17.180584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:17.180660 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:17.180761 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:17.181895 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.183018 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.184781 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:17.185535 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:17.185663 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:17.190206 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:17.190213 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:17.190217 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:17.190257 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:17.195977 (+ 5720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.196935 (+ 958us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.196940 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.196944 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.197098 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:17.197561 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:17.197566 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:17.197569 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:17.198029 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:17.198178 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:17.200343 (+ 2165us) server_negotiation.cc:299] Negotiation successful
0811 12:12:17.200863 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:12:17.202575 27508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:17.204645 27508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:17.207159 27344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: Bootstrap starting.
W20260811 12:12:17.207510 27151 tablet.cc:2369] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:17.213040 27344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:17.215131 27344 log.cc:824] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:17.220456 27344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: No bootstrap required, opened a new log
I20260811 12:12:17.226897 27185 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [term 1 FOLLOWER]: Refusing update from remote peer 96bb3219f84444bea2afa6b32512148f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:17.227964 27505 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:17.243176 27344 raft_consensus.cc:348] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.244099 27344 raft_consensus.cc:374] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:17.244366 27344 raft_consensus.cc:729] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f919d2ffac874fddb5fc6c701928c431, State: Initialized, Role: FOLLOWER
I20260811 12:12:17.245612 27344 consensus_queue.cc:260] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } }
I20260811 12:12:17.252805 27489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bb3219f84444bea2afa6b32512148f. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.253119 27489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.256639 27510 mvcc.cc:205] Tried to move back new op lower bound from 7317300581251260416 to 7317300581063639040. Current Snapshot: MvccSnapshot[applied={T|T < 7317300581251260416}]
I20260811 12:12:17.260182 27470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.260495 27470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:17.260275 27511 mvcc.cc:205] Tried to move back new op lower bound from 7317300581251260416 to 7317300581063639040. Current Snapshot: MvccSnapshot[applied={T|T < 7317300581251260416}]
I20260811 12:12:17.262128 27508 catalog_manager.cc:1347] Generated new cluster ID: 0de38e944ca7484d83c1cf0851fb6496
I20260811 12:12:17.262224 27508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:17.264194 27489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.264430 27489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.267108 27302 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96bb3219f84444bea2afa6b32512148f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f919d2ffac874fddb5fc6c701928c431"
I20260811 12:12:17.267318 27303 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "96bb3219f84444bea2afa6b32512148f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f919d2ffac874fddb5fc6c701928c431" is_pre_election: true
I20260811 12:12:17.267916 27344 sys_catalog.cc:564] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:17.267590 27300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:17.278410 27300 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [term 1 FOLLOWER]: Refusing update from remote peer 96bb3219f84444bea2afa6b32512148f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:17.279228 27505 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:17.279485 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.279853 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.282234 27508 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:12:17.283545 27508 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:17.295658 27508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f: Generated new TSK 0
I20260811 12:12:17.296602 27508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:12:17.302467 27470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.302757 27470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96bb3219f84444bea2afa6b32512148f [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:17.309300 27514 sys_catalog.cc:455] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96bb3219f84444bea2afa6b32512148f. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.309671 27514 sys_catalog.cc:458] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.309309 27535 sys_catalog.cc:455] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "96bb3219f84444bea2afa6b32512148f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3754b467081c4054beedb10412faffaa" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46457 } } peers { permanent_uuid: "f919d2ffac874fddb5fc6c701928c431" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 45603 } } peers { permanent_uuid: "96bb3219f84444bea2afa6b32512148f" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 36039 } } }
I20260811 12:12:17.309803 27535 sys_catalog.cc:458] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:12:17.388125 27553 catalog_manager.cc:1259] Loaded cluster ID: 0de38e944ca7484d83c1cf0851fb6496
I20260811 12:12:17.388250 27553 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: loading cluster ID for follower catalog manager: success
I20260811 12:12:17.391826 27553 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: acquiring CA information for follower catalog manager: success
I20260811 12:12:17.394366 27553 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f919d2ffac874fddb5fc6c701928c431: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:12:18.159930 27503 catalog_manager.cc:1259] Loaded cluster ID: 0de38e944ca7484d83c1cf0851fb6496
I20260811 12:12:18.160028 27503 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: loading cluster ID for follower catalog manager: success
I20260811 12:12:18.162437 27503 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: acquiring CA information for follower catalog manager: success
I20260811 12:12:18.164213 27503 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3754b467081c4054beedb10412faffaa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:12:18.465050 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27118
I20260811 12:12:18.466383 27215 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:18.573927 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27118
I20260811 12:12:18.599057 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27237
I20260811 12:12:18.600147 27334 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:18.703974 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27237
I20260811 12:12:18.731971 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27356
I20260811 12:12:18.733172 27443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:18.839109 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27356
2026-08-11T12:12:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4759 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:12:18 dist-test-slave-nj21 krb5kdc[27567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:12:18 dist-test-slave-nj21 krb5kdc[27567](info): set up 2 sockets
Aug 11 12:12:18 dist-test-slave-nj21 krb5kdc[27567](info): commencing operation
krb5kdc: starting...
W20260811 12:12:20.080056 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.001s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:12:20 dist-test-slave-nj21 krb5kdc[27567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:12:20.335139 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44999
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:44999
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:20.662317 27585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:20.662591 27585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:20.662721 27585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:20.669790 27585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:20.669891 27585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:20.669960 27585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:20.670045 27585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:20.670100 27585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:20.679747 27585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:44999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:20.681033 27585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:20.682331 27585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:20.693701 27592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:20.693642 27594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:20.693678 27598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:20.694769 27585 server_base.cc:1034] running on GCE node
I20260811 12:12:20.695497 27585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:20.696906 27585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:20.698091 27585 hybrid_clock.cc:648] HybridClock initialized: now 1786450340698039 us; error 43 us; skew 500 ppm
I20260811 12:12:20.698406 27585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:20 dist-test-slave-nj21 krb5kdc[27567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450340, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:20.701334 27585 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:20.702723 27585 webserver.cc:460] Webserver started at http://127.4.48.254:42075/ using document root <none> and password file <none>
I20260811 12:12:20.703363 27585 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:20.703473 27585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:20.703809 27585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:20.706775 27585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "7fe08a2677994db8826757d704dee14e"
format_stamp: "Formatted at 2026-08-11 12:12:20 on dist-test-slave-nj21"
server_key: "9a492850d54edc03cb85a16c219d0a42"
server_key_iv: "314b2b1bd3bf93e256ec71391fc89626"
server_key_version: "encryptionkey@0"
I20260811 12:12:20.707536 27585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "7fe08a2677994db8826757d704dee14e"
format_stamp: "Formatted at 2026-08-11 12:12:20 on dist-test-slave-nj21"
server_key: "9a492850d54edc03cb85a16c219d0a42"
server_key_iv: "314b2b1bd3bf93e256ec71391fc89626"
server_key_version: "encryptionkey@0"
I20260811 12:12:20.712524 27585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 12:12:20.715664 27607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:20.716912 27585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:12:20.717067 27585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "7fe08a2677994db8826757d704dee14e"
format_stamp: "Formatted at 2026-08-11 12:12:20 on dist-test-slave-nj21"
server_key: "9a492850d54edc03cb85a16c219d0a42"
server_key_iv: "314b2b1bd3bf93e256ec71391fc89626"
server_key_version: "encryptionkey@0"
I20260811 12:12:20.717240 27585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:20.742156 27585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:20.746068 27585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:20.746366 27585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:20.764891 27585 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:44999
I20260811 12:12:20.764926 27687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:44999 every 8 connection(s)
I20260811 12:12:20.766299 27585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:20.769114 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27585
I20260811 12:12:20.769343 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:12:20.770181 4291 external_mini_cluster.cc:1428] Setting key b063027aff64f629e1af8b460bb72068
I20260811 12:12:20.770550 27688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:20.784102 27688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e: Bootstrap starting.
Aug 11 12:12:20 dist-test-slave-nj21 krb5kdc[27567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:20.787146 27688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:20.788231 27688 log.cc:824] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:20.790860 27688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e: No bootstrap required, opened a new log
I20260811 12:12:20.793041 27691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:20.771977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51472 (local address 127.4.48.254:44999)
0811 12:12:20.772698 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:20.772709 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:20.772749 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:12:20.774990 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:20.775012 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:20.775365 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:20.775699 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:20.777491 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:20.778870 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:20.780669 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:20.781356 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:20.781476 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:20.786171 (+ 4695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:20.786177 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:20.786191 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:20.786225 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:20.789202 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:20.789883 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:20.789890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:20.789896 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:20.790056 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:20.790420 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:20.790423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:20.790425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:20.790913 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:20.791208 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:20.791700 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 12:12:20.792082 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":314,"threads_started":1}
I20260811 12:12:20.796551 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:12:20.798548 27688 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } }
I20260811 12:12:20.798950 27688 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:20.799046 27688 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fe08a2677994db8826757d704dee14e, State: Initialized, Role: FOLLOWER
I20260811 12:12:20.799558 27688 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } }
I20260811 12:12:20.799873 27688 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:20.799999 27688 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:20.800150 27688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:20.802126 27688 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } }
I20260811 12:12:20.802577 27688 leader_election.cc:302] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7fe08a2677994db8826757d704dee14e; no voters:
I20260811 12:12:20.803076 27688 leader_election.cc:288] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:12:20.803192 27696 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:20.803822 27696 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [term 1 LEADER]: Becoming Leader. State: Replica: 7fe08a2677994db8826757d704dee14e, State: Running, Role: LEADER
I20260811 12:12:20.804344 27696 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } }
I20260811 12:12:20.804900 27688 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:20.807035 27698 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7fe08a2677994db8826757d704dee14e. Latest consensus state: current_term: 1 leader_uuid: "7fe08a2677994db8826757d704dee14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } } }
I20260811 12:12:20.807049 27697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7fe08a2677994db8826757d704dee14e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7fe08a2677994db8826757d704dee14e" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44999 } } }
I20260811 12:12:20.807500 27697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:20.807500 27698 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:20.809368 27706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:20.814538 27706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:20.822757 27706 catalog_manager.cc:1347] Generated new cluster ID: f99b89ca1655425884c718246b90170c
I20260811 12:12:20.822870 27706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:20.837066 27706 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:12:20.838290 27706 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:20.846704 27706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7fe08a2677994db8826757d704dee14e: Generated new TSK 0
I20260811 12:12:20.847429 27706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:12:20.856998 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27585
I20260811 12:12:20.858141 27680 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:20.945017 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27585
2026-08-11T12:12:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2104 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:12:21 dist-test-slave-nj21 krb5kdc[27732](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:12:21 dist-test-slave-nj21 krb5kdc[27732](info): set up 2 sockets
Aug 11 12:12:21 dist-test-slave-nj21 krb5kdc[27732](info): commencing operation
krb5kdc: starting...
W20260811 12:12:22.168921 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.002s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:12:22 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:12:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:12:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:12:22.411545 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:22.736387 27750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:22.736668 27750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:22.736763 27750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:22.743724 27750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:22.743818 27750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:22.743870 27750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:22.743924 27750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:22.743966 27750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:22.753333 27750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:22.754515 27750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:22.755731 27750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:22.767256 27757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:22.767215 27756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:22.767295 27760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:22.768368 27750 server_base.cc:1034] running on GCE node
I20260811 12:12:22.769081 27750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:22.770375 27750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:22.771574 27750 hybrid_clock.cc:648] HybridClock initialized: now 1786450342771530 us; error 32 us; skew 500 ppm
I20260811 12:12:22.771919 27750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:22 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:22.774895 27750 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:22.776314 27750 webserver.cc:460] Webserver started at http://127.4.48.254:34873/ using document root <none> and password file <none>
I20260811 12:12:22.776940 27750 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:22.777056 27750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:22.777333 27750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:22.780194 27750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "3e734984f1e34d3ab2b37b6c671f0119"
format_stamp: "Formatted at 2026-08-11 12:12:22 on dist-test-slave-nj21"
server_key: "4e1df2ff3fb9aa28567009ac393b9c5e"
server_key_iv: "600592a807f6c9543d4f05435408646a"
server_key_version: "encryptionkey@0"
I20260811 12:12:22.780923 27750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "3e734984f1e34d3ab2b37b6c671f0119"
format_stamp: "Formatted at 2026-08-11 12:12:22 on dist-test-slave-nj21"
server_key: "4e1df2ff3fb9aa28567009ac393b9c5e"
server_key_iv: "600592a807f6c9543d4f05435408646a"
server_key_version: "encryptionkey@0"
I20260811 12:12:22.785977 27750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:12:22.789326 27770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:22.790555 27750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:12:22.790717 27750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "3e734984f1e34d3ab2b37b6c671f0119"
format_stamp: "Formatted at 2026-08-11 12:12:22 on dist-test-slave-nj21"
server_key: "4e1df2ff3fb9aa28567009ac393b9c5e"
server_key_iv: "600592a807f6c9543d4f05435408646a"
server_key_version: "encryptionkey@0"
I20260811 12:12:22.790894 27750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:22.811825 27750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:22.815318 27750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:22.815562 27750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:22.831449 27750 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45431
I20260811 12:12:22.831481 27857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45431 every 8 connection(s)
I20260811 12:12:22.832762 27750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:22.834553 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27750
I20260811 12:12:22.834789 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:12:22.835664 4291 external_mini_cluster.cc:1428] Setting key 6437d8d5159380027c5a23861311b674
I20260811 12:12:22.837227 27858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:12:22 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:22.850610 27858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap starting.
I20260811 12:12:22.853714 27858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:22.854923 27858 log.cc:824] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:22.858145 27858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: No bootstrap required, opened a new log
I20260811 12:12:22.858305 27864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:22.837675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38422 (local address 127.4.48.254:45431)
0811 12:12:22.838420 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:22.838431 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:22.838471 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:12:22.840076 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:22.840097 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:22.840454 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:22.840766 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:22.842455 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:22.843802 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:22.845597 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:22.846221 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:22.846337 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:22.851186 (+ 4849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:22.851194 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:22.851206 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:22.851240 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:22.854096 (+ 2856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:22.854877 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:22.854882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:22.854888 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:22.855089 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:22.855512 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:22.855520 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:22.855523 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:22.856263 (+ 740us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:22.856527 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:22.856944 (+ 417us) server_negotiation.cc:299] Negotiation successful
0811 12:12:22.857242 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":345,"threads_started":1}
I20260811 12:12:22.865860 27858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:22.866183 27858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:22.866273 27858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Initialized, Role: FOLLOWER
I20260811 12:12:22.866799 27858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:22.867025 27858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:22.867161 27858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:22.867313 27858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:22.869391 27858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:22.869941 27858 leader_election.cc:302] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e734984f1e34d3ab2b37b6c671f0119; no voters:
I20260811 12:12:22.870523 27858 leader_election.cc:288] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:12:22.870647 27866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:22.871258 27866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 LEADER]: Becoming Leader. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Running, Role: LEADER
I20260811 12:12:22.871829 27866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:22.872444 27858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:22.874398 27869 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e734984f1e34d3ab2b37b6c671f0119. Latest consensus state: current_term: 1 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:22.874763 27869 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:22.875609 27881 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:22.875514 27867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:22.876662 27867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:22.880867 27881 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:22.888381 27881 catalog_manager.cc:1347] Generated new cluster ID: 01a95a74a62446d7aab9fac83d18ed2b
I20260811 12:12:22.888485 27881 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:22.910269 27881 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:12:22.911466 27881 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:22.919622 27881 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Generated new TSK 0
I20260811 12:12:22.920351 27881 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:12:22.984630 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:23.309682 27898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:23.309955 27898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:23.310098 27898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:23.317024 27898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:23.317138 27898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:23.317289 27898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:23.326473 27898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:23.327695 27898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:23.328980 27898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:23.341475 27910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:23.341398 27914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:23.341426 27911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:23.342115 27898 server_base.cc:1034] running on GCE node
I20260811 12:12:23.342634 27898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:23.343454 27898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:23.344636 27898 hybrid_clock.cc:648] HybridClock initialized: now 1786450343344566 us; error 60 us; skew 500 ppm
I20260811 12:12:23.344908 27898 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:23 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450343, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:23.347862 27898 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:23.349171 27898 webserver.cc:460] Webserver started at http://127.4.48.193:34063/ using document root <none> and password file <none>
I20260811 12:12:23.349793 27898 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:23.349897 27898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:23.350150 27898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:23.352944 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "6a87c9efcedc4daa921e53c15eb06556"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "b47e24fdc5135f3a63372bfc92908fb2"
server_key_iv: "9dcad5e8520620f83c25f9983f0b5309"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.353626 27898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "6a87c9efcedc4daa921e53c15eb06556"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "b47e24fdc5135f3a63372bfc92908fb2"
server_key_iv: "9dcad5e8520620f83c25f9983f0b5309"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.358923 27898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:12:23.362100 27923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.363327 27898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:12:23.363456 27898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "6a87c9efcedc4daa921e53c15eb06556"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "b47e24fdc5135f3a63372bfc92908fb2"
server_key_iv: "9dcad5e8520620f83c25f9983f0b5309"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.363603 27898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:23.378461 27898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:23.381994 27898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:23.382215 27898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:23.383038 27898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:23.384284 27898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:23.384356 27898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.384441 27898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:23.384487 27898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.413537 27898 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:38611
I20260811 12:12:23.413575 28100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:38611 every 8 connection(s)
I20260811 12:12:23.414724 27898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:23.420090 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 27898
I20260811 12:12:23.420317 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:12:23.421139 4291 external_mini_cluster.cc:1428] Setting key 9e540ed7ef397510491d01d6b8baa598
Aug 11 12:12:23 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450343, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:23.434974 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:23.417027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:53865 (local address 127.4.48.254:45431)
0811 12:12:23.417460 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:23.417465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:23.418378 (+ 913us) server_negotiation.cc:407] Connection header received
0811 12:12:23.420482 (+ 2104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:23.420486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:23.420552 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:23.420638 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:23.422494 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.423384 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.425463 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.425836 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.425924 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:23.429685 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:23.429690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:23.429694 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:23.429724 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:23.432123 (+ 2399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:23.432857 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:23.432861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:23.432863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:23.432968 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:23.433370 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:23.433373 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:23.433375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:23.433617 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:23.433720 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:23.434449 (+ 729us) server_negotiation.cc:299] Negotiation successful
0811 12:12:23.434652 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":213,"threads_started":1}
I20260811 12:12:23.435823 28105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:23.417546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:53865)
0811 12:12:23.418234 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:23.418271 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:23.420161 (+ 1890us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:23.420912 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:23.420924 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:23.421635 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:23.422336 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:23.422356 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.423537 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.423542 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:23.425336 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:23.425350 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.426037 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.426043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:23.426301 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:23.426922 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:23.426948 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:23.429481 (+ 2533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:23.432247 (+ 2766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:23.432256 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:23.432270 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:23.432754 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:23.433052 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:23.433056 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:23.433058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:23.433279 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:23.433724 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:23.433731 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:23.434095 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 12:12:23.434363 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 12:12:23.434762 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":270,"threads_started":1}
I20260811 12:12:23.437215 28101 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:23.437489 28101 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:23.438153 28101 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:23.440380 27798 ts_manager.cc:194] Registered new tserver with Master: 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:23.442204 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:53865
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:12:23.478348 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:23.813865 28109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:23.814148 28109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:23.814282 28109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:23.821445 28109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:23.821573 28109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:23.821772 28109 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:23.831317 28109 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:23.832599 28109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:23.834013 28109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:23.847990 28120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:23.848058 28124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:23.848026 28121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:23.848649 28109 server_base.cc:1034] running on GCE node
I20260811 12:12:23.849254 28109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:23.850139 28109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:23.851341 28109 hybrid_clock.cc:648] HybridClock initialized: now 1786450343851296 us; error 37 us; skew 500 ppm
I20260811 12:12:23.851631 28109 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:23 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450343, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:23.854548 28109 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:23.855887 28109 webserver.cc:460] Webserver started at http://127.4.48.194:44697/ using document root <none> and password file <none>
I20260811 12:12:23.856542 28109 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:23.856657 28109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:23.856936 28109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:23.859829 28109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "a545ad17bcf9494bac0b638c05c005c0"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "d8790f5db1858f8bbae5278d2797894c"
server_key_iv: "9dc02174fabb188477ac1894b9f5ab18"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.860562 28109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "a545ad17bcf9494bac0b638c05c005c0"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "d8790f5db1858f8bbae5278d2797894c"
server_key_iv: "9dc02174fabb188477ac1894b9f5ab18"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.865586 28109 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 12:12:23.869009 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.870289 28109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 12:12:23.870432 28109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "a545ad17bcf9494bac0b638c05c005c0"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "d8790f5db1858f8bbae5278d2797894c"
server_key_iv: "9dc02174fabb188477ac1894b9f5ab18"
server_key_version: "encryptionkey@0"
I20260811 12:12:23.870616 28109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:23.903936 28109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:23.907467 28109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:23.907747 28109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:23.908634 28109 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:23.909931 28109 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:23.910009 28109 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.910132 28109 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:23.910184 28109 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:23.940152 28109 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:39425
I20260811 12:12:23.940215 28305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:39425 every 8 connection(s)
I20260811 12:12:23.941377 28109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:23.948848 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 28109
I20260811 12:12:23.949012 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:12:23.949482 4291 external_mini_cluster.cc:1428] Setting key f25325779bafa5a190cf0da70dbda366
Aug 11 12:12:23 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450343, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:23.965692 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:23.943748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:42919 (local address 127.4.48.254:45431)
0811 12:12:23.944160 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:23.944164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:23.945066 (+ 902us) server_negotiation.cc:407] Connection header received
0811 12:12:23.946831 (+ 1765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:23.946837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:23.946917 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:23.947015 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:23.948512 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.949248 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.951199 (+ 1951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.951748 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.951854 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:23.960409 (+ 8555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:23.960414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:23.960417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:23.960449 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:23.962729 (+ 2280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:23.963398 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:23.963403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:23.963406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:23.963572 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:23.964088 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:23.964092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:23.964094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:23.964342 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:23.964449 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:23.965259 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 12:12:23.965416 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":213,"threads_started":1}
I20260811 12:12:23.966974 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:23.944237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:42919)
0811 12:12:23.944909 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:23.944953 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:23.946573 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:23.947253 (+ 680us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:23.947260 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:23.947785 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:23.948327 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:23.948344 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.949392 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.949396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:23.951007 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:23.951021 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:23.951978 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:23.951986 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:23.952289 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:23.957734 (+ 5445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:23.957767 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:23.960194 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:23.962857 (+ 2663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:23.962866 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:23.962880 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:23.963272 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:23.963736 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:23.963741 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:23.963744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:23.963974 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:23.964462 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:23.964469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:23.964883 (+ 414us) client_negotiation.cc:769] Sending connection context
0811 12:12:23.965207 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 12:12:23.965677 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":258,"threads_started":1}
I20260811 12:12:23.968309 28306 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:23.968578 28306 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:23.969250 28306 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:23.970606 27798 ts_manager.cc:194] Registered new tserver with Master: a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
I20260811 12:12:23.971375 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:42919
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:12:24.006520 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:24.324054 28314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:24.324323 28314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:24.324467 28314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:24.331413 28314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:24.331527 28314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:24.331704 28314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:24.340974 28314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:24.342142 28314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:24.343381 28314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:24.355755 28322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:24.355866 28323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:24.355798 28326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:24.356458 28314 server_base.cc:1034] running on GCE node
I20260811 12:12:24.357013 28314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:24.357864 28314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:24.359063 28314 hybrid_clock.cc:648] HybridClock initialized: now 1786450344359023 us; error 27 us; skew 500 ppm
I20260811 12:12:24.359345 28314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:24 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:24.362205 28314 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:24.363469 28314 webserver.cc:460] Webserver started at http://127.4.48.195:33983/ using document root <none> and password file <none>
I20260811 12:12:24.364151 28314 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:24.364270 28314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:24.364542 28314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:24.367306 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "a208c07ddfbc4f8798f4507f47c9b044"
format_stamp: "Formatted at 2026-08-11 12:12:24 on dist-test-slave-nj21"
server_key: "98396b1fe7bede14faef42f6636a21a4"
server_key_iv: "f98be29a744f89b4998533dc55c054d1"
server_key_version: "encryptionkey@0"
I20260811 12:12:24.368074 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "a208c07ddfbc4f8798f4507f47c9b044"
format_stamp: "Formatted at 2026-08-11 12:12:24 on dist-test-slave-nj21"
server_key: "98396b1fe7bede14faef42f6636a21a4"
server_key_iv: "f98be29a744f89b4998533dc55c054d1"
server_key_version: "encryptionkey@0"
I20260811 12:12:24.372920 28314 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:12:24.376325 28336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:24.377599 28314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:12:24.377750 28314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "a208c07ddfbc4f8798f4507f47c9b044"
format_stamp: "Formatted at 2026-08-11 12:12:24 on dist-test-slave-nj21"
server_key: "98396b1fe7bede14faef42f6636a21a4"
server_key_iv: "f98be29a744f89b4998533dc55c054d1"
server_key_version: "encryptionkey@0"
I20260811 12:12:24.377947 28314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:24.398602 28314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:24.402468 28314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:24.402729 28314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:24.403702 28314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:24.405010 28314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:24.405094 28314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:24.405211 28314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:24.405263 28314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:24.436105 28314 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:42003
I20260811 12:12:24.436152 28516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:42003 every 8 connection(s)
I20260811 12:12:24.437304 28314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:24.444890 28101 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
I20260811 12:12:24.447602 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 28314
I20260811 12:12:24.447793 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:12:24.448240 4291 external_mini_cluster.cc:1428] Setting key b2134135cd94f43ed0c568dc49400b8e
Aug 11 12:12:24 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450344, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:24.456404 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.439855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:42015 (local address 127.4.48.254:45431)
0811 12:12:24.440046 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.440050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.441107 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 12:12:24.443150 (+ 2043us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.443157 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.443260 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.443362 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:24.444912 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.445829 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.447500 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.448008 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.448136 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.451623 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:24.451627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:24.451631 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:24.451662 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:24.453977 (+ 2315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.454602 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.454606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.454608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.454715 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.455047 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.455050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.455051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.455292 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:24.455412 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.456019 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.456181 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:12:24.457249 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.440205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:42015)
0811 12:12:24.440938 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.440983 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.442871 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.443614 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.443622 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.444154 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:24.444729 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.444748 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.446002 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.446007 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.447350 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.447364 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.448254 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.448263 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.448512 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.449161 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:24.449190 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:24.451433 (+ 2243us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:24.454101 (+ 2668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:24.454107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:24.454118 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:24.454520 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:24.454787 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:24.454790 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:24.454792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:24.454972 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:24.455380 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:24.455386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:24.455659 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.455930 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.456278 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":282,"threads_started":1}
I20260811 12:12:24.458482 28519 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:24.458762 28519 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:24.459435 28519 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:24.460767 27798 ts_manager.cc:194] Registered new tserver with Master: a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003)
I20260811 12:12:24.461615 27798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:42015
I20260811 12:12:24.463487 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:12:24.479209 28308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.467859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38434 (local address 127.4.48.254:45431)
0811 12:12:24.468009 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.468012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.468446 (+ 434us) server_negotiation.cc:407] Connection header received
0811 12:12:24.468556 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.468560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.468623 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.468695 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:24.470024 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.470703 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.471959 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.472383 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.472467 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.474745 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:24.474749 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:24.474752 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:24.474781 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:24.477183 (+ 2402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.477784 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.477788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.477790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.477899 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.478156 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.478159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.478161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.478442 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:24.478631 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.478836 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.478977 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:12:24.482906 27797 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:12:24.489626 27797 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:12:24.535561 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.500798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39425 (local address 127.0.0.1:52330)
0811 12:12:24.526246 (+ 25448us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.526293 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.526518 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.528910 (+ 2392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.528915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.528938 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:24.529312 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.529323 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.531000 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.531005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.532722 (+ 1717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.532735 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534488 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.534494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.534693 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.534763 (+ 70us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.534902 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.535095 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25171,"mutex_wait_us":81,"thread_start_us":26905,"threads_started":1}
I20260811 12:12:24.535660 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.501058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52330 (local address 127.4.48.194:39425)
0811 12:12:24.528356 (+ 27298us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.528365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.528391 (+ 26us) server_negotiation.cc:407] Connection header received
0811 12:12:24.528481 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.528488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.528704 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.528904 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:24.529476 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.530866 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.532886 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534267 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.534419 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.534636 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.535173 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.535361 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27144,"thread_start_us":27013,"threads_started":1}
I20260811 12:12:24.535665 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.501081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:43054)
0811 12:12:24.526898 (+ 25817us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.526915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.527047 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.527762 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.527770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.527802 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:24.528256 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.528269 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.530375 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.530380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.532977 (+ 2597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.532992 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534758 (+ 1766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.534763 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.534935 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.534977 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.535041 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.535161 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25688,"mutex_wait_us":131,"thread_start_us":24748,"threads_started":1}
I20260811 12:12:24.536528 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.501973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36854 (local address 127.4.48.195:42003)
0811 12:12:24.526279 (+ 24306us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.526287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.527771 (+ 1484us) server_negotiation.cc:407] Connection header received
0811 12:12:24.528086 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.528093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.528312 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.528467 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:24.529377 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.530919 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.533388 (+ 2469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534712 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.534837 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.535061 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.536039 (+ 978us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.536203 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24145,"thread_start_us":24344,"threads_started":1}
I20260811 12:12:24.536744 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.501723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43054 (local address 127.4.48.193:38611)
0811 12:12:24.525427 (+ 23704us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.525438 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.527009 (+ 1571us) server_negotiation.cc:407] Connection header received
0811 12:12:24.527192 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.527200 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.527478 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.528354 (+ 876us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:24.528639 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.530211 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.533140 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534536 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.535807 (+ 1271us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.536229 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.536296 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.536450 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23484,"thread_start_us":23735,"threads_started":1}
I20260811 12:12:24.536852 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.501455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42003 (local address 127.0.0.1:36854)
0811 12:12:24.527646 (+ 26191us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.527661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.527866 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.528709 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.528712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.528731 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:24.529142 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.529158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.531092 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.531096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.533215 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.533224 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.534929 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.534934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.535105 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.535135 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.535208 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.535322 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26071,"mutex_wait_us":1,"thread_start_us":25678,"threads_started":1}
I20260811 12:12:24.540055 27996 tablet_service.cc:1467] Processing CreateTablet for tablet 461964b61f964e0e86a6d58870a2dac0 (DEFAULT_TABLE table=test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:24.540014 28206 tablet_service.cc:1467] Processing CreateTablet for tablet 461964b61f964e0e86a6d58870a2dac0 (DEFAULT_TABLE table=test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:24.540200 28409 tablet_service.cc:1467] Processing CreateTablet for tablet 461964b61f964e0e86a6d58870a2dac0 (DEFAULT_TABLE table=test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:24.541303 27996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461964b61f964e0e86a6d58870a2dac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:24.541388 28409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461964b61f964e0e86a6d58870a2dac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:24.541384 28206 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 461964b61f964e0e86a6d58870a2dac0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:24.552567 28550 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Bootstrap starting.
I20260811 12:12:24.554697 28552 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Bootstrap starting.
I20260811 12:12:24.556124 28550 tablet_bootstrap.cc:654] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:24.557390 28550 log.cc:824] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:24.558282 28552 tablet_bootstrap.cc:654] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:24.559360 28554 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Bootstrap starting.
I20260811 12:12:24.559583 28552 log.cc:824] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:24.560241 28550 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: No bootstrap required, opened a new log
I20260811 12:12:24.560560 28550 ts_tablet_manager.cc:1397] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:12:24.562544 28552 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: No bootstrap required, opened a new log
I20260811 12:12:24.562659 28554 tablet_bootstrap.cc:654] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:24.562813 28552 ts_tablet_manager.cc:1397] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 12:12:24.563870 28554 log.cc:824] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:24.566890 28554 tablet_bootstrap.cc:492] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: No bootstrap required, opened a new log
I20260811 12:12:24.567134 28554 ts_tablet_manager.cc:1397] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260811 12:12:24.569692 28550 raft_consensus.cc:348] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.570102 28550 raft_consensus.cc:374] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:24.570188 28550 raft_consensus.cc:729] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a87c9efcedc4daa921e53c15eb06556, State: Initialized, Role: FOLLOWER
I20260811 12:12:24.570775 28552 raft_consensus.cc:348] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.570868 28550 consensus_queue.cc:260] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.571086 28552 raft_consensus.cc:374] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:24.571174 28552 raft_consensus.cc:729] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a208c07ddfbc4f8798f4507f47c9b044, State: Initialized, Role: FOLLOWER
I20260811 12:12:24.572566 28550 ts_tablet_manager.cc:1428] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260811 12:12:24.572978 28552 consensus_queue.cc:260] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.577019 28554 raft_consensus.cc:348] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.577435 28554 raft_consensus.cc:374] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:24.577545 28554 raft_consensus.cc:729] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a545ad17bcf9494bac0b638c05c005c0, State: Initialized, Role: FOLLOWER
I20260811 12:12:24.578153 28554 consensus_queue.cc:260] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.578359 28519 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
I20260811 12:12:24.578974 28552 ts_tablet_manager.cc:1428] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260811 12:12:24.579269 28306 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
I20260811 12:12:24.579795 28554 ts_tablet_manager.cc:1428] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
W20260811 12:12:24.666952 28102 tablet.cc:2369] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:12:24.688776 28520 tablet.cc:2369] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:12:24.693193 28307 tablet.cc:2369] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:24.758453 28559 raft_consensus.cc:482] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:24.758759 28559 raft_consensus.cc:504] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.760723 28559 leader_election.cc:288] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425), 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:24.766991 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.761145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:47403 (local address 127.4.48.194:39425)
0811 12:12:24.761350 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.761356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.761374 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:12:24.761443 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.761448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.761524 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.761606 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:24.762105 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.763036 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.764977 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.766048 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.766165 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.766225 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.766594 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.766723 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:12:24.767043 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.760882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39425 (local address 127.4.48.195:47403)
0811 12:12:24.761168 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.761188 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.761305 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.761684 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.761689 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.761728 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:24.761936 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.761948 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.763176 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.763181 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.764799 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.764815 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.766189 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.766196 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.766412 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.766450 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.766564 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.766712 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":63}
I20260811 12:12:24.768091 28562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.760937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.4.48.195:38739)
0811 12:12:24.761875 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:24.761892 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:24.761986 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:24.762264 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:24.762269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:24.762295 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:24.762640 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.762652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.763864 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.763868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.765512 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:24.765526 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.767005 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.767012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:24.767518 (+ 506us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.767552 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:12:24.767734 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 12:12:24.767833 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":2,"thread_start_us":608,"threads_started":1}
I20260811 12:12:24.768146 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.761383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:38739 (local address 127.4.48.193:38611)
0811 12:12:24.761581 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.761587 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.761983 (+ 396us) server_negotiation.cc:407] Connection header received
0811 12:12:24.762099 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.762103 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.762165 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.762230 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:24.762776 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.763728 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.765658 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.766748 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.766885 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.766949 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.767777 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.767921 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:12:24.768119 28238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461964b61f964e0e86a6d58870a2dac0" candidate_uuid: "a208c07ddfbc4f8798f4507f47c9b044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a545ad17bcf9494bac0b638c05c005c0" is_pre_election: true
I20260811 12:12:24.768668 28238 raft_consensus.cc:2454] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a208c07ddfbc4f8798f4507f47c9b044 in term 0.
I20260811 12:12:24.769287 28338 leader_election.cc:302] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0; no voters:
I20260811 12:12:24.769675 28559 raft_consensus.cc:2781] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:12:24.769801 28559 raft_consensus.cc:482] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:12:24.769902 28559 raft_consensus.cc:3037] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:24.769843 28034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461964b61f964e0e86a6d58870a2dac0" candidate_uuid: "a208c07ddfbc4f8798f4507f47c9b044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a87c9efcedc4daa921e53c15eb06556" is_pre_election: true
I20260811 12:12:24.770337 28034 raft_consensus.cc:2454] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a208c07ddfbc4f8798f4507f47c9b044 in term 0.
I20260811 12:12:24.772321 28559 raft_consensus.cc:504] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.772884 28559 leader_election.cc:288] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [CANDIDATE]: Term 1 election: Requested vote from peers a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425), 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:24.773280 28238 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461964b61f964e0e86a6d58870a2dac0" candidate_uuid: "a208c07ddfbc4f8798f4507f47c9b044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a545ad17bcf9494bac0b638c05c005c0"
I20260811 12:12:24.773294 28034 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "461964b61f964e0e86a6d58870a2dac0" candidate_uuid: "a208c07ddfbc4f8798f4507f47c9b044" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a87c9efcedc4daa921e53c15eb06556"
I20260811 12:12:24.773454 28238 raft_consensus.cc:3037] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:24.773453 28034 raft_consensus.cc:3037] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:24.775713 28238 raft_consensus.cc:2454] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a208c07ddfbc4f8798f4507f47c9b044 in term 1.
I20260811 12:12:24.775727 28034 raft_consensus.cc:2454] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a208c07ddfbc4f8798f4507f47c9b044 in term 1.
I20260811 12:12:24.776091 28337 leader_election.cc:302] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556, a208c07ddfbc4f8798f4507f47c9b044; no voters:
I20260811 12:12:24.776340 28559 raft_consensus.cc:2781] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:24.776752 28559 raft_consensus.cc:686] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 1 LEADER]: Becoming Leader. State: Replica: a208c07ddfbc4f8798f4507f47c9b044, State: Running, Role: LEADER
I20260811 12:12:24.777206 28559 consensus_queue.cc:237] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } }
I20260811 12:12:24.781424 27797 catalog_manager.cc:5495] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 reported cstate change: term changed from 0 to 1, leader changed from <none> to a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "a208c07ddfbc4f8798f4507f47c9b044" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } health_report { overall_health: HEALTHY } } }
I20260811 12:12:24.829123 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.823813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.4.48.195:42003)
0811 12:12:24.824007 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.824012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.824035 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:12:24.824257 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.824261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.824364 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.824492 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:12:24.824877 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.825595 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.826979 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.827376 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.827485 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.828058 (+ 573us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:12:24.828572 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:12:24.828705 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.828800 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.828880 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:12:24.837698 28238 raft_consensus.cc:1261] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Refusing update from remote peer a208c07ddfbc4f8798f4507f47c9b044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:24.837698 28034 raft_consensus.cc:1261] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Refusing update from remote peer a208c07ddfbc4f8798f4507f47c9b044: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:24.838692 28563 consensus_queue.cc:1035] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:24.839021 28559 consensus_queue.cc:1035] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:24.852952 28568 mvcc.cc:205] Tried to move back new op lower bound from 7317300612444405760 to 7317300612210827264. Current Snapshot: MvccSnapshot[applied={T|T < 7317300612444405760}]
I20260811 12:12:24.866575 27798 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:24.866833 27798 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38434:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:24.870361 27798 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Sending DeleteTablet for 3 replicas of tablet 461964b61f964e0e86a6d58870a2dac0
I20260811 12:12:24.871425 27996 tablet_service.cc:1514] Processing DeleteTablet for tablet 461964b61f964e0e86a6d58870a2dac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:24 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:43054
I20260811 12:12:24.871431 28409 tablet_service.cc:1514] Processing DeleteTablet for tablet 461964b61f964e0e86a6d58870a2dac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:24 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:36854
I20260811 12:12:24.871438 28206 tablet_service.cc:1514] Processing DeleteTablet for tablet 461964b61f964e0e86a6d58870a2dac0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:24 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:52330
I20260811 12:12:24.872798 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27898
I20260811 12:12:24.877811 28585 tablet_replica.cc:330] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: stopping tablet replica
I20260811 12:12:24.878238 28585 raft_consensus.cc:2229] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:24.878734 28581 tablet_replica.cc:330] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: stopping tablet replica
I20260811 12:12:24.878724 28585 raft_consensus.cc:2258] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:24.879216 28581 raft_consensus.cc:2229] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:12:24.879915 28581 raft_consensus.cc:2258] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:24.880676 28579 tablet_replica.cc:330] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: stopping tablet replica
I20260811 12:12:24.881059 28579 raft_consensus.cc:2229] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:24.881608 28579 raft_consensus.cc:2258] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:24.883570 28579 ts_tablet_manager.cc:1905] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:24.884267 28581 ts_tablet_manager.cc:1905] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:12:24 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:12:24.890666 28581 ts_tablet_manager.cc:1918] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:24.890744 28579 ts_tablet_manager.cc:1918] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:24.890851 28581 log.cc:1196] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/461964b61f964e0e86a6d58870a2dac0
I20260811 12:12:24.890895 28579 log.cc:1196] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/461964b61f964e0e86a6d58870a2dac0
I20260811 12:12:24.891399 28581 ts_tablet_manager.cc:1939] T 461964b61f964e0e86a6d58870a2dac0 P a208c07ddfbc4f8798f4507f47c9b044: Deleting consensus metadata
I20260811 12:12:24.891443 28579 ts_tablet_manager.cc:1939] T 461964b61f964e0e86a6d58870a2dac0 P 6a87c9efcedc4daa921e53c15eb06556: Deleting consensus metadata
I20260811 12:12:24.893352 28585 ts_tablet_manager.cc:1905] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:24.893563 27774 catalog_manager.cc:4841] TS 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611): tablet 461964b61f964e0e86a6d58870a2dac0 (table test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]) successfully deleted
I20260811 12:12:24.893947 27782 catalog_manager.cc:4841] TS a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): tablet 461964b61f964e0e86a6d58870a2dac0 (table test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]) successfully deleted
I20260811 12:12:24.898442 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:24.874053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.4.48.193:38611)
0811 12:12:24.874225 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:24.874232 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:24.878109 (+ 3877us) server_negotiation.cc:407] Connection header received
0811 12:12:24.878290 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:24.878296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:24.878416 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:24.878518 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:24.880881 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.882058 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.883552 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:24.884201 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:24.884326 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:24.891542 (+ 7216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:24.891558 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:24.891575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:24.891614 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:24.895318 (+ 3704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.896143 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.896152 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.896156 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.896323 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:24.896755 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:24.896760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:24.896764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:24.897215 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:24.897516 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:24.897840 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 12:12:24.898069 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:12:24.898974 28585 ts_tablet_manager.cc:1918] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:24.899120 28585 log.cc:1196] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/461964b61f964e0e86a6d58870a2dac0
I20260811 12:12:24.899142 28095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:24.899602 28585 ts_tablet_manager.cc:1939] T 461964b61f964e0e86a6d58870a2dac0 P a545ad17bcf9494bac0b638c05c005c0: Deleting consensus metadata
I20260811 12:12:24.901326 27776 catalog_manager.cc:4841] TS a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): tablet 461964b61f964e0e86a6d58870a2dac0 (table test-table [id=3a2d24cb7f554fb08c8ffa392378b4f1]) successfully deleted
I20260811 12:12:25.001101 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27898
I20260811 12:12:25.025032 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 28109
Aug 11 12:12:25 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:12:25.041188 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:25.025779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52802 (local address 127.4.48.194:39425)
0811 12:12:25.026043 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:25.026048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:25.026069 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:12:25.026247 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:25.026253 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:25.026350 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:25.026436 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:25.027776 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:25.028512 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:25.029818 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:25.030189 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:25.030297 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:25.034718 (+ 4421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:25.034737 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:25.034758 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:25.034801 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:25.038530 (+ 3729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:25.039134 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:25.039144 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:25.039148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:25.039288 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:25.039598 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:25.039603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:25.039607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:25.040106 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:25.040397 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:25.040597 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 12:12:25.040822 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144}
I20260811 12:12:25.041893 28300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:25.130764 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 28109
I20260811 12:12:25.153923 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 28314
Aug 11 12:12:25 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:12:25.170060 28549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:25.154789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.4.48.195:42003)
0811 12:12:25.154993 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:25.155002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:25.155023 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:12:25.155088 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:25.155093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:25.155156 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:25.155243 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:25.156626 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:25.157483 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:25.158818 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:25.159278 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:25.159391 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:25.163935 (+ 4544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:25.163951 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:25.163969 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:25.164011 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:25.167044 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:25.167885 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:25.167894 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:25.167898 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:25.168086 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:25.168561 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:25.168565 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:25.168567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:25.168932 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:25.169173 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:25.169551 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 12:12:25.169713 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:12:25.170756 28508 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:25.290536 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 28314
I20260811 12:12:25.314752 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 27750
I20260811 12:12:25.315887 27851 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:25.422356 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 27750
I20260811 12:12:25.448380 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--webserver_interface=127.4.48.254
--webserver_port=34873
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:25.776125 28594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:25.776398 28594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:25.776523 28594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:25.783504 28594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:25.783607 28594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:25.783699 28594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:25.783780 28594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:25.783838 28594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:25.793725 28594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=34873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:25.794950 28594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:25.796244 28594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:25.819497 28602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:25.819497 28601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:25.819583 28605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:25.820456 28594 server_base.cc:1034] running on GCE node
I20260811 12:12:25.821360 28594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:25.822911 28594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:25.824124 28594 hybrid_clock.cc:648] HybridClock initialized: now 1786450345824072 us; error 34 us; skew 500 ppm
I20260811 12:12:25.824512 28594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:25 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450345, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:25.828029 28594 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:25.829661 28594 webserver.cc:460] Webserver started at http://127.4.48.254:34873/ using document root <none> and password file <none>
I20260811 12:12:25.830447 28594 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:25.830590 28594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:25.836467 28594 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:12:25.839275 28617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:25.840660 28594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:12:25.840850 28594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "3e734984f1e34d3ab2b37b6c671f0119"
format_stamp: "Formatted at 2026-08-11 12:12:22 on dist-test-slave-nj21"
server_key: "4e1df2ff3fb9aa28567009ac393b9c5e"
server_key_iv: "600592a807f6c9543d4f05435408646a"
server_key_version: "encryptionkey@0"
I20260811 12:12:25.841877 28594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:25.868857 28594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:25.873044 28594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:25.873288 28594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:25.890959 28594 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45431
I20260811 12:12:25.890995 28696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45431 every 8 connection(s)
I20260811 12:12:25.892374 28594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:25.899135 28697 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:12:25.901607 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 28594
I20260811 12:12:25.902629 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:38611
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=34063
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:25.903187 28697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap starting.
I20260811 12:12:25.925149 28697 log.cc:824] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:25.937426 28697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:12:25.937942 28697 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap complete.
I20260811 12:12:25.947082 28697 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:25.947923 28697 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Initialized, Role: FOLLOWER
I20260811 12:12:25.948524 28697 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:25.948755 28697 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:25.948879 28697 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:25.949046 28697 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:12:25.952807 28697 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:25.953299 28697 leader_election.cc:302] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e734984f1e34d3ab2b37b6c671f0119; no voters:
I20260811 12:12:25.953856 28697 leader_election.cc:288] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:12:25.953981 28704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:12:25.955060 28704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 LEADER]: Becoming Leader. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Running, Role: LEADER
I20260811 12:12:25.955602 28704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:25.956118 28697 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:25.958042 28707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:25.958434 28707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:25.962167 28709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e734984f1e34d3ab2b37b6c671f0119. Latest consensus state: current_term: 2 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:25.962491 28709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:25.964079 28713 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:25.972477 28713 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:25.973912 28713 catalog_manager.cc:1259] Loaded cluster ID: 01a95a74a62446d7aab9fac83d18ed2b
I20260811 12:12:25.974009 28713 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:25.977370 28713 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 12:12:25.996917 28713 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 12:12:25.998040 28732 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:12:26.004232 28713 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:26.005517 28713 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Loaded TSK: 0
I20260811 12:12:26.006603 28713 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:12:26.265290 28700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:26.265662 28700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:26.265848 28700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:26.272923 28700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:26.273031 28700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:26.273190 28700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:26.282572 28700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:38611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=34063
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:26.283885 28700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:26.285175 28700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:26.299322 28743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:26.299420 28747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:26.299983 28745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:26.300643 28700 server_base.cc:1034] running on GCE node
I20260811 12:12:26.301225 28700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:26.302177 28700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:26.303397 28700 hybrid_clock.cc:648] HybridClock initialized: now 1786450346303345 us; error 45 us; skew 500 ppm
I20260811 12:12:26.303718 28700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:26 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:26.306756 28700 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:26.308252 28700 webserver.cc:460] Webserver started at http://127.4.48.193:34063/ using document root <none> and password file <none>
I20260811 12:12:26.308923 28700 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:26.309042 28700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:26.314360 28700 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:12:26.317075 28760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.318365 28700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:12:26.318514 28700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "6a87c9efcedc4daa921e53c15eb06556"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "b47e24fdc5135f3a63372bfc92908fb2"
server_key_iv: "9dcad5e8520620f83c25f9983f0b5309"
server_key_version: "encryptionkey@0"
I20260811 12:12:26.319437 28700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:26.336551 28700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:26.340268 28700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:26.340502 28700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:26.341420 28700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:26.342686 28700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:26.342798 28700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.342906 28700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:26.342979 28700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.373219 28700 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:38611
I20260811 12:12:26.373286 28940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:38611 every 8 connection(s)
I20260811 12:12:26.374610 28700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:26.381424 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 28700
I20260811 12:12:26.382068 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:39425
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=44697
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:26 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:26.397341 28946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:26.377583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:36563)
0811 12:12:26.378346 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:26.378389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:26.380016 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:26.381331 (+ 1315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:26.381342 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:26.381937 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:26.382513 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:26.382532 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.384149 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.384153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:26.385329 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:26.385342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.386398 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.386404 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:26.386689 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:26.387392 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:26.387422 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:26.390048 (+ 2626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:26.393583 (+ 3535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:26.393591 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:26.393606 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:26.394032 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:26.394476 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:26.394479 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:26.394481 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:26.394655 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:26.395467 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:26.395473 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:26.395803 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 12:12:26.396029 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 12:12:26.396379 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":328,"threads_started":1}
I20260811 12:12:26.397774 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:26.377324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:36563 (local address 127.4.48.254:45431)
0811 12:12:26.377914 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:26.377931 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:26.378527 (+ 596us) server_negotiation.cc:407] Connection header received
0811 12:12:26.380497 (+ 1970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:26.380509 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:26.380818 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:26.381139 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:26.382735 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.383986 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.385472 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.386154 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.386317 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:26.390332 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:26.390342 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:26.390360 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:26.390404 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:26.393423 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:26.394166 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:26.394174 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:26.394181 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:26.394375 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:26.394751 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:26.394756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:26.394760 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:26.395349 (+ 589us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:26.395543 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:26.396283 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 12:12:26.396666 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":224,"threads_started":1}
I20260811 12:12:26.400440 28941 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:26.400758 28941 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:26.401572 28941 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:26.403908 28638 ts_manager.cc:194] Registered new tserver with Master: 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:26.405392 28638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:36563
W20260811 12:12:26.711602 28947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:26.711871 28947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:26.712045 28947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:26.719051 28947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:26.719154 28947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:26.719300 28947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:26.728590 28947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:39425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=44697
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:26.729820 28947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:26.731150 28947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:26.743610 28955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:26.743688 28958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:26.743891 28961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:26.744510 28947 server_base.cc:1034] running on GCE node
I20260811 12:12:26.745103 28947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:26.745952 28947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:26.747151 28947 hybrid_clock.cc:648] HybridClock initialized: now 1786450346747103 us; error 54 us; skew 500 ppm
I20260811 12:12:26.747452 28947 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:26 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:26.750432 28947 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:26.751777 28947 webserver.cc:460] Webserver started at http://127.4.48.194:44697/ using document root <none> and password file <none>
I20260811 12:12:26.752429 28947 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:26.752553 28947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:26.757560 28947 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 12:12:26.760176 28976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.761458 28947 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 12:12:26.761611 28947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "a545ad17bcf9494bac0b638c05c005c0"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "d8790f5db1858f8bbae5278d2797894c"
server_key_iv: "9dc02174fabb188477ac1894b9f5ab18"
server_key_version: "encryptionkey@0"
I20260811 12:12:26.762584 28947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:26.780654 28947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:26.784447 28947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:26.784655 28947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:26.785509 28947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:26.786707 28947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:26.786794 28947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.786875 28947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:26.786927 28947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:26.816546 28947 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:39425
I20260811 12:12:26.816660 29144 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:39425 every 8 connection(s)
I20260811 12:12:26.817943 28947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:26.822129 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 28947
I20260811 12:12:26.822944 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:42003
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=33983
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:12:26 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450346, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:26.838750 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:26.820310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:47787 (local address 127.4.48.254:45431)
0811 12:12:26.820523 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:26.820529 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:26.821663 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 12:12:26.824095 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:26.824100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:26.824169 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:26.824255 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:26.826102 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.826759 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.828869 (+ 2110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.829278 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.829372 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:26.833144 (+ 3772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:26.833149 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:26.833152 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:26.833185 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:26.835893 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:26.836569 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:26.836574 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:26.836578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:26.836724 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:26.837099 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:26.837103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:26.837106 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:26.837441 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:26.837570 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:26.838219 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 12:12:26.838448 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:12:26.839459 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:26.820789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:47787)
0811 12:12:26.821486 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:26.821533 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:26.823810 (+ 2277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:26.824537 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:26.824548 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:26.825221 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:26.825953 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:26.825976 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.826934 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.826940 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:26.828741 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:26.828756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:26.829504 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:26.829511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:26.829776 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:26.830414 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:26.830441 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:26.832938 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:26.836012 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:26.836018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:26.836029 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:26.836439 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:26.836819 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:26.836823 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:26.836825 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:26.836994 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:26.837557 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:26.837563 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:26.837848 (+ 285us) client_negotiation.cc:769] Sending connection context
0811 12:12:26.838063 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 12:12:26.838422 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":276,"threads_started":1}
I20260811 12:12:26.840817 29145 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:26.841123 29145 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:26.841969 29145 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:26.843457 28638 ts_manager.cc:194] Registered new tserver with Master: a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
I20260811 12:12:26.844260 28638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:47787
W20260811 12:12:27.151341 29149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:27.151602 29149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:27.151763 29149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:27.158778 29149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:27.158879 29149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:27.159024 29149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:27.168583 29149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:42003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=33983
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:27.169821 29149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:27.171069 29149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:27.183473 29159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:27.183432 29164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:27.183499 29168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:27.184535 29149 server_base.cc:1034] running on GCE node
I20260811 12:12:27.185099 29149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:27.185925 29149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:27.187112 29149 hybrid_clock.cc:648] HybridClock initialized: now 1786450347187062 us; error 37 us; skew 500 ppm
I20260811 12:12:27.187376 29149 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:27 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:27.190250 29149 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:27.191572 29149 webserver.cc:460] Webserver started at http://127.4.48.195:33983/ using document root <none> and password file <none>
I20260811 12:12:27.192256 29149 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:27.192369 29149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:27.197316 29149 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:12:27.199822 29178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:27.201046 29149 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:27.201196 29149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "a208c07ddfbc4f8798f4507f47c9b044"
format_stamp: "Formatted at 2026-08-11 12:12:24 on dist-test-slave-nj21"
server_key: "98396b1fe7bede14faef42f6636a21a4"
server_key_iv: "f98be29a744f89b4998533dc55c054d1"
server_key_version: "encryptionkey@0"
I20260811 12:12:27.202082 29149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:27.214182 29149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:27.217701 29149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:27.217957 29149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:27.218816 29149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:27.220062 29149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:27.220155 29149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:27.220253 29149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:27.220321 29149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:27.249977 29149 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:42003
I20260811 12:12:27.250041 29345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:42003 every 8 connection(s)
I20260811 12:12:27.251367 29149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:27.252920 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 29149
Aug 11 12:12:27 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450347, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:27.269595 29352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.253934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36228 (local address 127.4.48.254:45431)
0811 12:12:27.254435 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.254440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.254800 (+ 360us) server_negotiation.cc:407] Connection header received
0811 12:12:27.255003 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.255008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.255089 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.255203 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:27.257220 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.258134 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.259962 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.260584 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.260790 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.263709 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:27.263715 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:27.263741 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:27.263779 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:27.266775 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.267536 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.267541 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.267544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.267746 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.268119 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.268122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.268124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.268488 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:27.268755 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.268959 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.269151 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":103,"server-negotiator.queue_time_us":408,"thread_start_us":234,"threads_started":1}
I20260811 12:12:27.273322 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.253930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:60257 (local address 127.4.48.254:45431)
0811 12:12:27.254099 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.254105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.255391 (+ 1286us) server_negotiation.cc:407] Connection header received
0811 12:12:27.257795 (+ 2404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.257800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.257873 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.257967 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:27.259722 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.260794 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.262946 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.263370 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.263456 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.267355 (+ 3899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:27.267359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:27.267362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:27.267390 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:27.270550 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.271245 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.271249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.271251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.271371 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.272036 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.272040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.272042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.272284 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:27.272386 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.272952 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.273088 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":46}
I20260811 12:12:27.274228 29353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.254486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:60257)
0811 12:12:27.255232 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.255277 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.257567 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.258138 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.258145 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.258809 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:27.259527 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.259550 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.260949 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.260955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.262810 (+ 1855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.262825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.263757 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.263765 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.264012 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.264641 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:27.264672 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:27.267169 (+ 2497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:27.270667 (+ 3498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:27.270676 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:27.270690 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:27.271134 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:27.271472 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:27.271477 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:27.271481 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:27.271742 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:27.272391 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:27.272397 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:27.272672 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.272884 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.273213 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":296,"threads_started":1}
I20260811 12:12:27.275332 29346 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45431
I20260811 12:12:27.275607 29346 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:27.276288 29346 heartbeater.cc:511] Master 127.4.48.254:45431 requested a full tablet report, sending...
I20260811 12:12:27.277542 28638 ts_manager.cc:194] Registered new tserver with Master: a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003)
I20260811 12:12:27.278298 28638 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:60257
I20260811 12:12:27.283892 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:12:27.290020 28944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.284757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36234 (local address 127.4.48.254:45431)
0811 12:12:27.284947 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.284952 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.285252 (+ 300us) server_negotiation.cc:407] Connection header received
0811 12:12:27.285406 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.285423 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.285533 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.285636 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:12:27.286001 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.286680 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.288082 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.288475 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.288559 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.289126 (+ 567us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:12:27.289550 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:12:27.289646 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.289774 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.289828 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:12:27.290793 28638 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:12:27.297936 28638 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:12:27.320673 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.308807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42003 (local address 127.0.0.1:56750)
0811 12:12:27.309761 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.309800 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.309974 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.312792 (+ 2818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.312800 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.312833 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:27.313248 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.313260 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.315457 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.315463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.317866 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.317882 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.319772 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.319779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.319975 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.320029 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.320128 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.320305 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"threads_started":1}
I20260811 12:12:27.322064 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.309001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56750 (local address 127.4.48.195:42003)
0811 12:12:27.312172 (+ 3171us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.312180 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.312211 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:12:27.312309 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.312316 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.312540 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.313477 (+ 937us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:27.313745 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.315297 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.318117 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.319505 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.321248 (+ 1743us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.321488 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.321561 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.321714 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3028,"thread_start_us":2871,"threads_started":1}
I20260811 12:12:27.327489 29243 tablet_service.cc:1467] Processing CreateTablet for tablet 67f842e90d184eed887d8ce7e70789af (DEFAULT_TABLE table=test-table [id=d9bbeb1bd2934b5fb8c0a75245b94d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:27.329041 29243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f842e90d184eed887d8ce7e70789af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:27.330041 29366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.308979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39425 (local address 127.0.0.1:52818)
0811 12:12:27.320890 (+ 11911us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.320908 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.321005 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.323623 (+ 2618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.323628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.323648 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:27.323848 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.323859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.325677 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.325682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.327476 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.327507 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.329279 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.329285 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.329475 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.329518 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.329607 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.329729 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11803,"mutex_wait_us":24,"thread_start_us":949,"threads_started":1}
I20260811 12:12:27.330199 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.309709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52818 (local address 127.4.48.194:39425)
0811 12:12:27.323068 (+ 13359us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.323076 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.323108 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:12:27.323207 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.323214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.323426 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.324019 (+ 593us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:27.324276 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.325534 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.327652 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.329064 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.329259 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.329503 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.329685 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.329841 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13219,"thread_start_us":13075,"threads_started":1}
I20260811 12:12:27.331125 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.312023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:57958)
0811 12:12:27.323420 (+ 11397us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.323437 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.323623 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.324613 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.324618 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.324643 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:27.324985 (+ 342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.324997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.326543 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.326547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.328184 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.328198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.330385 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.330394 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.330558 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.330589 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.330664 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.330792 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11262,"mutex_wait_us":1,"thread_start_us":11086,"threads_started":1}
I20260811 12:12:27.331903 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.312418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.4.48.193:38611)
0811 12:12:27.322649 (+ 10231us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.322658 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.323596 (+ 938us) server_negotiation.cc:407] Connection header received
0811 12:12:27.324025 (+ 429us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.324032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.324416 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.324613 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:27.325146 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.326417 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.328351 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.329787 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.329909 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.330086 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.331425 (+ 1339us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.331550 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10097,"thread_start_us":10035,"threads_started":1}
I20260811 12:12:27.333777 29044 tablet_service.cc:1467] Processing CreateTablet for tablet 67f842e90d184eed887d8ce7e70789af (DEFAULT_TABLE table=test-table [id=d9bbeb1bd2934b5fb8c0a75245b94d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:27.334671 28834 tablet_service.cc:1467] Processing CreateTablet for tablet 67f842e90d184eed887d8ce7e70789af (DEFAULT_TABLE table=test-table [id=d9bbeb1bd2934b5fb8c0a75245b94d09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:27.335210 29044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f842e90d184eed887d8ce7e70789af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:27.335728 28834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67f842e90d184eed887d8ce7e70789af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:27.344372 29376 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Bootstrap starting.
I20260811 12:12:27.348074 29376 tablet_bootstrap.cc:654] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:27.348374 29377 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Bootstrap starting.
I20260811 12:12:27.349462 29376 log.cc:824] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:27.349687 29379 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Bootstrap starting.
I20260811 12:12:27.351366 29377 tablet_bootstrap.cc:654] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:27.352291 29377 log.cc:824] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:27.353199 29379 tablet_bootstrap.cc:654] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:27.354393 29379 log.cc:824] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:27.355055 29377 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: No bootstrap required, opened a new log
I20260811 12:12:27.355324 29377 ts_tablet_manager.cc:1397] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260811 12:12:27.357071 29379 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: No bootstrap required, opened a new log
I20260811 12:12:27.357295 29379 ts_tablet_manager.cc:1397] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:12:27.361311 29376 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: No bootstrap required, opened a new log
I20260811 12:12:27.361560 29376 ts_tablet_manager.cc:1397] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.011s
I20260811 12:12:27.362455 29377 raft_consensus.cc:348] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.362890 29377 raft_consensus.cc:374] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:27.362995 29377 raft_consensus.cc:729] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a87c9efcedc4daa921e53c15eb06556, State: Initialized, Role: FOLLOWER
I20260811 12:12:27.363541 29377 consensus_queue.cc:260] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.364624 28941 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
I20260811 12:12:27.365324 29377 ts_tablet_manager.cc:1428] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Time spent starting tablet: real 0.010s user 0.011s sys 0.000s
I20260811 12:12:27.365816 29379 raft_consensus.cc:348] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.366276 29379 raft_consensus.cc:374] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:27.366376 29379 raft_consensus.cc:729] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a545ad17bcf9494bac0b638c05c005c0, State: Initialized, Role: FOLLOWER
I20260811 12:12:27.366932 29379 consensus_queue.cc:260] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.372262 29145 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
I20260811 12:12:27.372061 29376 raft_consensus.cc:348] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.372941 29379 ts_tablet_manager.cc:1428] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Time spent starting tablet: real 0.016s user 0.010s sys 0.006s
I20260811 12:12:27.373191 29376 raft_consensus.cc:374] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:27.373306 29376 raft_consensus.cc:729] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a208c07ddfbc4f8798f4507f47c9b044, State: Initialized, Role: FOLLOWER
I20260811 12:12:27.373842 29376 consensus_queue.cc:260] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.375355 29376 ts_tablet_manager.cc:1428] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Time spent starting tablet: real 0.014s user 0.013s sys 0.001s
I20260811 12:12:27.375358 29346 heartbeater.cc:503] Master 127.4.48.254:45431 was elected leader, sending a full tablet report...
W20260811 12:12:27.376708 28943 tablet.cc:2369] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:27.412200 29386 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:27.412484 29386 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.414144 29386 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:27.420819 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.414529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:59837 (local address 127.4.48.195:42003)
0811 12:12:27.414718 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.414724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.415423 (+ 699us) server_negotiation.cc:407] Connection header received
0811 12:12:27.415593 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.415597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.415655 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.415768 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:27.416406 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.417182 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.419074 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.419892 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.420047 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.420121 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.420449 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.420576 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:12:27.420921 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.414527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42003 (local address 127.4.48.194:59837)
0811 12:12:27.415310 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.415325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.415453 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.415790 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.415794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.415816 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:27.416233 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.416249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.417309 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.417315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.418926 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.418938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.420122 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.420129 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.420287 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.420321 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.420397 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.420481 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":81,"thread_start_us":449,"threads_started":1}
I20260811 12:12:27.421379 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.414629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:49493 (local address 127.4.48.193:38611)
0811 12:12:27.414816 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.414821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.415524 (+ 703us) server_negotiation.cc:407] Connection header received
0811 12:12:27.415719 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.415725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.415790 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.415882 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:27.416586 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.417309 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.419143 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.420054 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.420196 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.420267 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.421023 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.421147 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:12:27.421995 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.414608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.4.48.194:49493)
0811 12:12:27.415420 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.415438 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:27.415567 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:27.415899 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:27.415903 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:27.415941 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:27.416422 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.416434 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.417460 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.417463 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.418983 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:27.418997 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.420607 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.420616 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:27.420804 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.420852 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 12:12:27.421535 (+ 683us) client_negotiation.cc:240] Negotiation successful
0811 12:12:27.421657 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":68,"thread_start_us":436,"threads_started":1}
I20260811 12:12:27.421877 29270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f842e90d184eed887d8ce7e70789af" candidate_uuid: "a545ad17bcf9494bac0b638c05c005c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a208c07ddfbc4f8798f4507f47c9b044" is_pre_election: true
I20260811 12:12:27.422394 29270 raft_consensus.cc:2454] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a545ad17bcf9494bac0b638c05c005c0 in term 0.
I20260811 12:12:27.423024 28981 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0; no voters:
I20260811 12:12:27.422977 28861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f842e90d184eed887d8ce7e70789af" candidate_uuid: "a545ad17bcf9494bac0b638c05c005c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a87c9efcedc4daa921e53c15eb06556" is_pre_election: true
I20260811 12:12:27.423362 29386 raft_consensus.cc:2781] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:12:27.423519 29386 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:12:27.423487 28861 raft_consensus.cc:2454] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a545ad17bcf9494bac0b638c05c005c0 in term 0.
I20260811 12:12:27.423640 29386 raft_consensus.cc:3037] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:27.425901 29386 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.426386 29386 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [CANDIDATE]: Term 1 election: Requested vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611)
I20260811 12:12:27.426767 28861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f842e90d184eed887d8ce7e70789af" candidate_uuid: "a545ad17bcf9494bac0b638c05c005c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6a87c9efcedc4daa921e53c15eb06556"
I20260811 12:12:27.426774 29270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67f842e90d184eed887d8ce7e70789af" candidate_uuid: "a545ad17bcf9494bac0b638c05c005c0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a208c07ddfbc4f8798f4507f47c9b044"
I20260811 12:12:27.426925 28861 raft_consensus.cc:3037] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:27.426923 29270 raft_consensus.cc:3037] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:27.428987 28861 raft_consensus.cc:2454] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a545ad17bcf9494bac0b638c05c005c0 in term 1.
I20260811 12:12:27.429123 29270 raft_consensus.cc:2454] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a545ad17bcf9494bac0b638c05c005c0 in term 1.
I20260811 12:12:27.429394 28977 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556, a545ad17bcf9494bac0b638c05c005c0; no voters:
I20260811 12:12:27.429652 29386 raft_consensus.cc:2781] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:27.430058 29386 raft_consensus.cc:686] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 1 LEADER]: Becoming Leader. State: Replica: a545ad17bcf9494bac0b638c05c005c0, State: Running, Role: LEADER
I20260811 12:12:27.430531 29386 consensus_queue.cc:237] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:27.434691 28638 catalog_manager.cc:5495] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 reported cstate change: term changed from 0 to 1, leader changed from <none> to a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194). New cstate: current_term: 1 leader_uuid: "a545ad17bcf9494bac0b638c05c005c0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } health_report { overall_health: UNKNOWN } } }
I20260811 12:12:27.455852 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.450739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52822 (local address 127.4.48.194:39425)
0811 12:12:27.450930 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.450935 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.450954 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:12:27.451134 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.451138 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.451206 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.451331 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:12:27.451720 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.452390 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.453847 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.454245 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.454352 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.454839 (+ 487us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:12:27.455286 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:12:27.455410 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.455517 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.455592 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:12:27.463344 29270 raft_consensus.cc:1261] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Refusing update from remote peer a545ad17bcf9494bac0b638c05c005c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:27.463476 28861 raft_consensus.cc:1261] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Refusing update from remote peer a545ad17bcf9494bac0b638c05c005c0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:27.464433 29390 consensus_queue.cc:1035] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:27.464802 29386 consensus_queue.cc:1035] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:27.475570 29400 mvcc.cc:205] Tried to move back new op lower bound from 7317300623200296960 to 7317300623079448576. Current Snapshot: MvccSnapshot[applied={T|T < 7317300623200296960}]
I20260811 12:12:27.486686 28638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:27.486974 28638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:27.490455 28638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Sending DeleteTablet for 3 replicas of tablet 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:27.491477 29243 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f842e90d184eed887d8ce7e70789af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:27 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:56750
I20260811 12:12:27.491833 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 28700
I20260811 12:12:27.492018 29044 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f842e90d184eed887d8ce7e70789af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:27 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:52818
I20260811 12:12:27.492626 28834 tablet_service.cc:1514] Processing DeleteTablet for tablet 67f842e90d184eed887d8ce7e70789af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:27 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:57958
I20260811 12:12:27.501770 29416 tablet_replica.cc:330] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: stopping tablet replica
I20260811 12:12:27.502202 29416 raft_consensus.cc:2229] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:27.502697 29416 raft_consensus.cc:2258] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:27.504402 29412 tablet_replica.cc:330] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: stopping tablet replica
I20260811 12:12:27.504706 29412 raft_consensus.cc:2229] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:27.505434 29412 raft_consensus.cc:2258] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:27.507272 29415 tablet_replica.cc:330] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: stopping tablet replica
I20260811 12:12:27.507434 29412 ts_tablet_manager.cc:1905] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:27.507702 29415 raft_consensus.cc:2229] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:12:27.506863 29416 ts_tablet_manager.cc:1905] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:27.508358 29415 raft_consensus.cc:2258] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:27.510183 29415 ts_tablet_manager.cc:1905] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:27.510850 29373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.493456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.4.48.193:38611)
0811 12:12:27.493637 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.493644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.493724 (+ 80us) server_negotiation.cc:407] Connection header received
0811 12:12:27.493921 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.493928 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.494028 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.494142 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:27.495966 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.497219 (+ 1253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.499225 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.499949 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.500074 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.504050 (+ 3976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:27.504067 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:27.504086 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:27.504126 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:27.507845 (+ 3719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.508636 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.508644 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.508648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.508802 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.509183 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.509188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.509191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.509719 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:27.510049 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.510309 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.510518 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:12:27.511497 28936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:27.517058 29412 ts_tablet_manager.cc:1918] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:27.517277 29412 log.cc:1196] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/67f842e90d184eed887d8ce7e70789af
I20260811 12:12:27.517922 29412 ts_tablet_manager.cc:1939] T 67f842e90d184eed887d8ce7e70789af P a208c07ddfbc4f8798f4507f47c9b044: Deleting consensus metadata
I20260811 12:12:27.518934 29415 ts_tablet_manager.cc:1918] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:27.519121 29415 log.cc:1196] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/67f842e90d184eed887d8ce7e70789af
I20260811 12:12:27.519712 29415 ts_tablet_manager.cc:1939] T 67f842e90d184eed887d8ce7e70789af P a545ad17bcf9494bac0b638c05c005c0: Deleting consensus metadata
I20260811 12:12:27.520459 28621 catalog_manager.cc:4841] TS a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): tablet 67f842e90d184eed887d8ce7e70789af (table test-table [id=d9bbeb1bd2934b5fb8c0a75245b94d09]) successfully deleted
I20260811 12:12:27.521417 28619 catalog_manager.cc:4841] TS a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): tablet 67f842e90d184eed887d8ce7e70789af (table test-table [id=d9bbeb1bd2934b5fb8c0a75245b94d09]) successfully deleted
I20260811 12:12:27.609673 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 28700
I20260811 12:12:27.636053 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 28947
W20260811 12:12:27.636102 28618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.48.193:38611 (error 108)
I20260811 12:12:27.638159 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.637688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:57986)
0811 12:12:27.637863 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.637979 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 12:12:27.638420 28618 catalog_manager.cc:4584] TS 6a87c9efcedc4daa921e53c15eb06556 (127.4.48.193:38611): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 67f842e90d184eed887d8ce7e70789af: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
I20260811 12:12:27.652011 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.636884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52830 (local address 127.4.48.194:39425)
0811 12:12:27.637072 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.637084 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.637845 (+ 761us) server_negotiation.cc:407] Connection header received
0811 12:12:27.637922 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.637928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.638014 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.638142 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:27.639859 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.640621 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.642535 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.642957 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.643042 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.646343 (+ 3301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:27.646356 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:27.646372 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:27.646411 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:27.649265 (+ 2854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.650058 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.650064 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.650067 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.650211 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.650584 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.650588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.650590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.650933 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:27.651171 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.651494 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.651645 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:12:27.652765 29138 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:27.692315 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.691742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:57990)
0811 12:12:27.691995 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.692122 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:12:27.693466 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.693056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:57994)
0811 12:12:27.693209 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.693297 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 12:12:27.754122 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.753623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:58008)
0811 12:12:27.753852 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.753948 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:12:27.755175 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.754858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:58018)
0811 12:12:27.754983 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.755054 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 12:12:27.777328 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 28947
I20260811 12:12:27.800526 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 29149
I20260811 12:12:27.813000 29369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.801267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56766 (local address 127.4.48.195:42003)
0811 12:12:27.801439 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:27.801443 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:27.801463 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:12:27.801607 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:27.801611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:27.801676 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:27.801756 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:27.802981 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.803732 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.805045 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:27.805441 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:27.805551 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:27.807976 (+ 2425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:27.807989 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:27.808006 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:27.808052 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:27.810654 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.811245 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.811251 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.811254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.811391 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:27.811644 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:27.811647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:27.811648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:27.812039 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:27.812243 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:27.812511 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 12:12:27.812681 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:12:27.813588 29341 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:27.835566 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.835024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:58024)
0811 12:12:27.835251 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.835366 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 12:12:27.836673 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:27.836217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:58030)
0811 12:12:27.836440 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:27.836540 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 12:12:27.905205 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 29149
I20260811 12:12:27.927070 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 28594
I20260811 12:12:27.928341 28692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:28.041764 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:28.041006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:38611 (local address 127.0.0.1:58038)
0811 12:12:28.041197 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:28.041315 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:38611: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:12:28.041993 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 28594
I20260811 12:12:28.071386 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--webserver_interface=127.4.48.254
--webserver_port=34873
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45431
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:28.402379 29425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:28.402624 29425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:28.402712 29425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:28.409668 29425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:28.409771 29425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:28.409821 29425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:28.409884 29425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:28.409926 29425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:28.419258 29425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=34873
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:28.420457 29425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:28.421674 29425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:28.432729 29437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:28.432731 29434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:28.433279 29433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:28.433722 29425 server_base.cc:1034] running on GCE node
I20260811 12:12:28.434502 29425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:28.435883 29425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:28.437106 29425 hybrid_clock.cc:648] HybridClock initialized: now 1786450348437043 us; error 39 us; skew 500 ppm
I20260811 12:12:28.437458 29425 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:28 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450348, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:28.440552 29425 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:28.441988 29425 webserver.cc:460] Webserver started at http://127.4.48.254:34873/ using document root <none> and password file <none>
I20260811 12:12:28.442693 29425 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:28.442826 29425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:28.448254 29425 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:12:28.450968 29445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:28.452255 29425 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:28.452441 29425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "3e734984f1e34d3ab2b37b6c671f0119"
format_stamp: "Formatted at 2026-08-11 12:12:22 on dist-test-slave-nj21"
server_key: "4e1df2ff3fb9aa28567009ac393b9c5e"
server_key_iv: "600592a807f6c9543d4f05435408646a"
server_key_version: "encryptionkey@0"
I20260811 12:12:28.453351 29425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:28.484191 29425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:28.488171 29425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:28.488435 29425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:28.505344 29425 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45431
I20260811 12:12:28.505319 29544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45431 every 8 connection(s)
I20260811 12:12:28.506690 29425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:28.513288 29545 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:12:28.515815 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 29425
I20260811 12:12:28.516551 29545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap starting.
I20260811 12:12:28.516817 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:38611
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=34063
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:28.539367 29545 log.cc:824] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:28.560812 29545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:12:28.561420 29545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119: Bootstrap complete.
I20260811 12:12:28.574795 29545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:28.575982 29545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Initialized, Role: FOLLOWER
I20260811 12:12:28.576684 29545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:28.576956 29545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:28.577072 29545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:28.577260 29545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:12:28.582682 29545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:28.583254 29545 leader_election.cc:302] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3e734984f1e34d3ab2b37b6c671f0119; no voters:
I20260811 12:12:28.583936 29545 leader_election.cc:288] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:12:28.584055 29551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:12:28.585170 29551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [term 3 LEADER]: Becoming Leader. State: Replica: 3e734984f1e34d3ab2b37b6c671f0119, State: Running, Role: LEADER
I20260811 12:12:28.585646 29551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } }
I20260811 12:12:28.586011 29545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:28.587841 29554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3e734984f1e34d3ab2b37b6c671f0119. Latest consensus state: current_term: 3 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:28.588202 29554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:28.588733 29553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "3e734984f1e34d3ab2b37b6c671f0119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3e734984f1e34d3ab2b37b6c671f0119" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45431 } } }
I20260811 12:12:28.589023 29553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3e734984f1e34d3ab2b37b6c671f0119 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:28.597566 29559 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:28.606396 29559 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:28.607946 29559 catalog_manager.cc:1259] Loaded cluster ID: 01a95a74a62446d7aab9fac83d18ed2b
I20260811 12:12:28.608049 29559 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 12:12:28.610337 29559 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f64fb0479ad google::LogMessage::Fail() at ??:0
@ 0x7f64fb04a88a google::LogMessage::SendToLog() at ??:0
@ 0x7f64fb0474de google::LogMessage::Flush() at ??:0
@ 0x7f64fb048a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6510a7e512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f6510a7b976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6510b03610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f6510b03439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f64fdc93345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f64fdc9691c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f64fdc96799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f64fdc71eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6500f4b6db start_thread at ??:0
@ 0x7f64f9ee161f clone at ??:0
W20260811 12:12:28.903155 29549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:28.903425 29549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:28.903561 29549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:28.910892 29549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:28.911010 29549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:28.911168 29549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:28.920706 29549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:38611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=34063
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:28.921917 29549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:28.923224 29549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:28.936544 29605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:28.936617 29606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:28.936645 29608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:28.937556 29549 server_base.cc:1034] running on GCE node
I20260811 12:12:28.938133 29549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:28.939049 29549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:28.940253 29549 hybrid_clock.cc:648] HybridClock initialized: now 1786450348940195 us; error 46 us; skew 500 ppm
I20260811 12:12:28.940572 29549 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:28 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450348, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:28.943495 29549 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:28.944893 29549 webserver.cc:460] Webserver started at http://127.4.48.193:34063/ using document root <none> and password file <none>
I20260811 12:12:28.945533 29549 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:28.945649 29549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:28.950687 29549 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 12:12:28.953526 29617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:28.954720 29549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 12:12:28.954873 29549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "6a87c9efcedc4daa921e53c15eb06556"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "b47e24fdc5135f3a63372bfc92908fb2"
server_key_iv: "9dcad5e8520620f83c25f9983f0b5309"
server_key_version: "encryptionkey@0"
I20260811 12:12:28.955798 29549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:28.974257 29549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:28.977789 29549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:28.978039 29549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:28.978942 29549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:28.980759 29632 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:12:28.986209 29549 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:12:28.986334 29549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260811 12:12:28.986443 29549 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:12:28.989554 29549 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:12:28.989631 29549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.001s sys 0.001s
I20260811 12:12:28.989977 29632 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Bootstrap starting.
I20260811 12:12:28.997995 29632 log.cc:824] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:29.011749 29632 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:12:29.012470 29632 tablet_bootstrap.cc:492] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Bootstrap complete.
I20260811 12:12:29.013052 29632 ts_tablet_manager.cc:1397] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Time spent bootstrapping tablet: real 0.023s user 0.010s sys 0.012s
I20260811 12:12:29.022646 29632 raft_consensus.cc:348] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:29.024116 29632 raft_consensus.cc:729] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a87c9efcedc4daa921e53c15eb06556, State: Initialized, Role: FOLLOWER
I20260811 12:12:29.024786 29632 consensus_queue.cc:260] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:29.026302 29632 ts_tablet_manager.cc:1428] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556: Time spent starting tablet: real 0.013s user 0.013s sys 0.001s
I20260811 12:12:29.029934 29549 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:38611
I20260811 12:12:29.030014 29798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:38611 every 8 connection(s)
I20260811 12:12:29.031066 29549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:29.032305 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 29549
I20260811 12:12:29.033077 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:39425
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=44697
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:12:29.036124 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.033896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:40003)
0811 12:12:29.034789 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.035034 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":368,"threads_started":1}
W20260811 12:12:29.036803 29799 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:45431 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:45431: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
I20260811 12:12:29.037634 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.037287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:37603)
0811 12:12:29.037436 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.037514 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:12:29.038528 29803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.038154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:59861)
0811 12:12:29.038291 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.038389 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:12:29.038765 29799 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:12:29.364326 29805 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:29.364610 29805 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:29.364756 29805 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:29.371807 29805 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:29.371903 29805 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:29.372042 29805 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:29.381188 29805 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:39425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=44697
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:29.382369 29805 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:29.383615 29805 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:29.396059 29822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:29.396011 29814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:29.396082 29815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:29.396766 29805 server_base.cc:1034] running on GCE node
I20260811 12:12:29.397317 29805 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:29.398172 29805 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:29.399364 29805 hybrid_clock.cc:648] HybridClock initialized: now 1786450349399321 us; error 33 us; skew 500 ppm
I20260811 12:12:29.399648 29805 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:29 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450349, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:29.402619 29805 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:29.404013 29805 webserver.cc:460] Webserver started at http://127.4.48.194:44697/ using document root <none> and password file <none>
I20260811 12:12:29.404649 29805 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:29.404757 29805 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:29.409750 29805 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 12:12:29.412276 29833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.413497 29805 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:29.413650 29805 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "a545ad17bcf9494bac0b638c05c005c0"
format_stamp: "Formatted at 2026-08-11 12:12:23 on dist-test-slave-nj21"
server_key: "d8790f5db1858f8bbae5278d2797894c"
server_key_iv: "9dc02174fabb188477ac1894b9f5ab18"
server_key_version: "encryptionkey@0"
I20260811 12:12:29.414574 29805 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:29.445888 29805 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:29.449604 29805 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:29.449858 29805 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:29.450819 29805 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:29.452153 29805 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:29.452251 29805 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.452360 29805 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:29.452430 29805 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.482318 29805 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:39425
I20260811 12:12:29.482393 30022 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:39425 every 8 connection(s)
I20260811 12:12:29.483795 29805 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:29.488447 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.486547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:48609)
0811 12:12:29.487301 (+ 754us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.487562 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":264,"threads_started":1}
W20260811 12:12:29.489056 30026 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:45431 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:45431: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
I20260811 12:12:29.489759 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.489479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:51679)
0811 12:12:29.489596 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.489659 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 12:12:29.490461 30029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.490186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:48097)
0811 12:12:29.490284 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.490352 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 12:12:29.490658 30026 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:12:29.492359 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 29805
I20260811 12:12:29.492976 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:42003
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:45431
--webserver_port=33983
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:29.821419 30031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:29.821703 30031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:29.821812 30031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:29.828914 30031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:29.829025 30031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:29.829149 30031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:29.838652 30031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:42003
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=33983
--tserver_master_addrs=127.4.48.254:45431
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:29.839886 30031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:29.841197 30031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:29.853724 30041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:29.853777 30042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:29.853781 30045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:29.854521 30031 server_base.cc:1034] running on GCE node
I20260811 12:12:29.855085 30031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:29.855944 30031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:29.857136 30031 hybrid_clock.cc:648] HybridClock initialized: now 1786450349857086 us; error 41 us; skew 500 ppm
I20260811 12:12:29.857424 30031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:29 dist-test-slave-nj21 krb5kdc[27732](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450349, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:29.860491 30031 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:29.861835 30031 webserver.cc:460] Webserver started at http://127.4.48.195:33983/ using document root <none> and password file <none>
I20260811 12:12:29.862490 30031 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:29.862599 30031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:29.867658 30031 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:29.870236 30053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.871431 30031 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 12:12:29.871574 30031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "a208c07ddfbc4f8798f4507f47c9b044"
format_stamp: "Formatted at 2026-08-11 12:12:24 on dist-test-slave-nj21"
server_key: "98396b1fe7bede14faef42f6636a21a4"
server_key_iv: "f98be29a744f89b4998533dc55c054d1"
server_key_version: "encryptionkey@0"
I20260811 12:12:29.872519 30031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:29.900722 30031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:29.904275 30031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:29.904524 30031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:29.905438 30031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:29.906720 30031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:29.906807 30031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.906917 30031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:29.906981 30031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:29.937686 30031 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:42003
I20260811 12:12:29.937744 30245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:42003 every 8 connection(s)
I20260811 12:12:29.939111 30031 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:29.943643 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.941869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:46881)
0811 12:12:29.942578 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.942802 (+ 224us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":271,"threads_started":1}
W20260811 12:12:29.944289 30247 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:45431 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:45431: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
I20260811 12:12:29.945029 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.944716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:53223)
0811 12:12:29.944849 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.944914 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:12:29.945820 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:29.945507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:40487)
0811 12:12:29.945642 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:29.945715 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:12:29.946070 30247 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:12:29.947891 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 30031
I20260811 12:12:30.039954 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.039221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:39247)
0811 12:12:30.039628 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:30.039736 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":204,"threads_started":1}
I20260811 12:12:30.231940 30256 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:30.232378 30256 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:30.234146 30256 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
Aug 11 12:12:30 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450348, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
Aug 11 12:12:30 dist-test-slave-nj21 krb5kdc[27732](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450348, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:12:30.255877 30255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.234410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:42003 (local address 127.4.48.193:50549)
0811 12:12:30.235387 (+ 977us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:30.235409 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:30.235714 (+ 305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:30.238290 (+ 2576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:30.238294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:30.238901 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:30.239647 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:30.239665 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.241187 (+ 1522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.241191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:30.242640 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:30.242654 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.244019 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.244025 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:30.244350 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:30.245178 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:30.245202 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:30.248119 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:30.252316 (+ 4197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:30.252322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:30.252333 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:30.252735 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:30.253152 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:30.253158 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:30.253161 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:30.253413 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:30.254297 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:30.254305 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:30.254673 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 12:12:30.255011 (+ 338us) client_negotiation.cc:240] Negotiation successful
0811 12:12:30.255415 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":175,"spinlock_wait_cycles":50560}
I20260811 12:12:30.256052 30258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.234631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:50549 (local address 127.4.48.195:42003)
0811 12:12:30.236146 (+ 1515us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:30.236156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:30.236191 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:12:30.237612 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:30.237620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:30.237975 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:30.238312 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:30.239856 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.241018 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.242819 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.243794 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.243978 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:30.248403 (+ 4425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:30.248414 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:30.248431 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:30.248476 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:30.252185 (+ 3709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:30.252859 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:30.252865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:30.252871 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:30.253026 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:30.253550 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:30.253555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:30.253559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:30.254187 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:30.254329 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:30.255240 (+ 911us) server_negotiation.cc:299] Negotiation successful
0811 12:12:30.255648 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1364,"thread_start_us":1206,"threads_started":1}
I20260811 12:12:30.256017 30257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.234429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39425 (local address 127.4.48.193:38081)
0811 12:12:30.235080 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:30.235119 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:30.235705 (+ 586us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:30.238186 (+ 2481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:30.238197 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:30.238917 (+ 720us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:30.239647 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:30.239666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.241195 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.241200 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:30.242640 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:30.242652 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.244002 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.244009 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:30.244361 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:30.245162 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:30.245192 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:30.248323 (+ 3131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:30.252833 (+ 4510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:30.252836 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:30.252838 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:30.253176 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:30.253948 (+ 772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:30.253952 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:30.253956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:30.254114 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:30.255000 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:30.255005 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:30.255162 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 12:12:30.255301 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 12:12:30.255562 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":197,"spinlock_wait_cycles":29568,"thread_start_us":328,"threads_started":1}
I20260811 12:12:30.256381 30260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.234842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:38081 (local address 127.4.48.194:39425)
0811 12:12:30.235329 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:30.235344 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:30.235477 (+ 133us) server_negotiation.cc:407] Connection header received
0811 12:12:30.237409 (+ 1932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:30.237418 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:30.237755 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:30.237997 (+ 242us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:30.239864 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.241006 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.242786 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:30.243794 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:30.243978 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:30.248581 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:30.248591 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:30.248605 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:30.248647 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:30.252167 (+ 3520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:30.253341 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:30.253349 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:30.253356 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:30.253566 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:30.254229 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:30.254235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:30.254238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:30.254855 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:30.255043 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:30.255576 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 12:12:30.255981 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":208,"threads_started":1}
W20260811 12:12:30.259470 29624 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
W20260811 12:12:30.260249 29620 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:30.260366 29620 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556; no voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0
I20260811 12:12:30.260715 30256 raft_consensus.cc:2726] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:12:30.492040 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.491122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:44039)
0811 12:12:30.491614 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:30.491792 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":261,"threads_started":1}
I20260811 12:12:30.947289 30262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:30.946555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:54183)
0811 12:12:30.946942 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:30.947057 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":186,"threads_started":1}
I20260811 12:12:31.041502 30263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:31.040838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:38563)
0811 12:12:31.041265 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:31.041343 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":224,"threads_started":1}
I20260811 12:12:31.493587 30264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:31.492854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:48763)
0811 12:12:31.493328 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:31.493418 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":249,"threads_started":1}
I20260811 12:12:31.851181 30266 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:31.851401 30266 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:31.852068 30266 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
W20260811 12:12:31.853077 29624 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
W20260811 12:12:31.853245 29620 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:31.853371 29620 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556; no voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0
I20260811 12:12:31.853631 30266 raft_consensus.cc:2726] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:12:31.948858 30267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:31.948113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:34193)
0811 12:12:31.948603 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:31.948682 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":244,"threads_started":1}
I20260811 12:12:32.043037 30269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:32.042282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:57849)
0811 12:12:32.042746 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:32.042833 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":222,"threads_started":1}
I20260811 12:12:32.495036 30270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:32.494383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:41963)
0811 12:12:32.494789 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:32.494864 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":211,"threads_started":1}
I20260811 12:12:32.950316 30271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:32.949615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:45485)
0811 12:12:32.950073 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:32.950159 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":251,"threads_started":1}
I20260811 12:12:33.044559 30273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:33.043894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:52521)
0811 12:12:33.044331 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:33.044409 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":235,"threads_started":1}
I20260811 12:12:33.496577 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:33.495889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:54229)
0811 12:12:33.496310 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:33.496390 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":206,"threads_started":1}
I20260811 12:12:33.532936 30276 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:33.533150 30276 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:33.533962 30276 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
W20260811 12:12:33.534873 29620 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
W20260811 12:12:33.535199 29624 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:33.535338 29624 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556; no voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0
I20260811 12:12:33.535573 30276 raft_consensus.cc:2726] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:12:33.951838 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:33.951097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:39873)
0811 12:12:33.951500 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:33.951585 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":185,"threads_started":1}
I20260811 12:12:34.046155 30282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:34.045445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:42747)
0811 12:12:34.045884 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:34.045961 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":186,"threads_started":1}
I20260811 12:12:34.498046 30283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:34.497382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:37681)
0811 12:12:34.497799 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:34.497876 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260811 12:12:34.953374 30285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:34.952664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:38415)
0811 12:12:34.953117 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:34.953205 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":235,"threads_started":1}
I20260811 12:12:35.047628 30287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:35.046952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:54093)
0811 12:12:35.047370 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:35.047459 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":212,"threads_started":1}
I20260811 12:12:35.499567 30288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:35.498836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:43699)
0811 12:12:35.499285 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:35.499365 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":232,"threads_started":1}
I20260811 12:12:35.954838 30290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:35.954152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:46209)
0811 12:12:35.954587 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:35.954665 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":233,"threads_started":1}
I20260811 12:12:36.049160 30292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:36.048447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:37165)
0811 12:12:36.048831 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:36.048956 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":206,"threads_started":1}
I20260811 12:12:36.501281 30293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:36.500454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:41329)
0811 12:12:36.501013 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:36.501095 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":246,"threads_started":1}
I20260811 12:12:36.652555 30294 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:36.652750 30294 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:36.653419 30294 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
W20260811 12:12:36.654337 29624 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
W20260811 12:12:36.654497 29620 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:36.654639 29620 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556; no voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0
I20260811 12:12:36.654871 30294 raft_consensus.cc:2726] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:12:36.956532 30295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:36.955640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:33849)
0811 12:12:36.956235 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:36.956312 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":259,"threads_started":1}
I20260811 12:12:37.050678 30300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:37.049928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:53171)
0811 12:12:37.050396 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:37.050483 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":229,"threads_started":1}
I20260811 12:12:37.502737 30301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:37.502087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:33675)
0811 12:12:37.502482 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:37.502557 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":197,"threads_started":1}
I20260811 12:12:37.957971 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:37.957325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:56517)
0811 12:12:37.957714 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:37.957791 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":203,"threads_started":1}
I20260811 12:12:38.052137 30303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:38.051437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:42991)
0811 12:12:38.051881 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:38.051961 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":230,"threads_started":1}
I20260811 12:12:38.504335 30305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:38.503518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:46883)
0811 12:12:38.504042 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:38.504163 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":272,"threads_started":1}
I20260811 12:12:38.864034 30309 raft_consensus.cc:482] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:38.864228 30309 raft_consensus.cc:504] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a208c07ddfbc4f8798f4507f47c9b044" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 42003 } } peers { permanent_uuid: "a545ad17bcf9494bac0b638c05c005c0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39425 } } peers { permanent_uuid: "6a87c9efcedc4daa921e53c15eb06556" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 38611 } }
I20260811 12:12:38.864936 30309 leader_election.cc:288] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003), a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425)
W20260811 12:12:38.865823 29624 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a208c07ddfbc4f8798f4507f47c9b044 (127.4.48.195:42003): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
W20260811 12:12:38.865995 29620 leader_election.cc:341] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a545ad17bcf9494bac0b638c05c005c0 (127.4.48.194:39425): Not found: Tablet not found: 67f842e90d184eed887d8ce7e70789af
I20260811 12:12:38.866135 29620 leader_election.cc:302] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6a87c9efcedc4daa921e53c15eb06556; no voters: a208c07ddfbc4f8798f4507f47c9b044, a545ad17bcf9494bac0b638c05c005c0
I20260811 12:12:38.866389 30309 raft_consensus.cc:2726] T 67f842e90d184eed887d8ce7e70789af P 6a87c9efcedc4daa921e53c15eb06556 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:12:38.959493 30310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:38.958761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:57223)
0811 12:12:38.959228 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:38.959338 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":231,"threads_started":1}
I20260811 12:12:39.053854 30311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:39.053094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:54013)
0811 12:12:39.053570 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:39.053663 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":224,"threads_started":1}
I20260811 12:12:39.505797 30313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:39.505154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:33555)
0811 12:12:39.505542 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:39.505620 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":201,"threads_started":1}
I20260811 12:12:39.961067 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:39.960410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:52519)
0811 12:12:39.960814 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:39.960896 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":208,"threads_started":1}
I20260811 12:12:40.055411 30316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:40.054718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:44553)
0811 12:12:40.055151 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:40.055242 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":228,"threads_started":1}
I20260811 12:12:40.507285 30317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:40.506577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:49617)
0811 12:12:40.507039 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:40.507131 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":224,"threads_started":1}
I20260811 12:12:40.962599 30318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:40.961865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:47753)
0811 12:12:40.962331 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:40.962414 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":214,"threads_started":1}
I20260811 12:12:41.056973 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:41.056232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:50939)
0811 12:12:41.056678 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:41.056796 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":236,"threads_started":1}
I20260811 12:12:41.508798 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:41.508105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:60779)
0811 12:12:41.508544 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:41.508622 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":232,"threads_started":1}
I20260811 12:12:41.964237 30329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:41.963515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:44663)
0811 12:12:41.963985 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:41.964065 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":253,"threads_started":1}
I20260811 12:12:42.058550 30330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:42.057790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:45831)
0811 12:12:42.058262 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:42.058353 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260811 12:12:42.510346 30332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:42.509590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:38047)
0811 12:12:42.510071 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:42.510163 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":240,"threads_started":1}
I20260811 12:12:42.965798 30333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:42.965024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:40147)
0811 12:12:42.965515 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:42.965629 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":238,"threads_started":1}
I20260811 12:12:43.060251 30335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:43.059404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:38797)
0811 12:12:43.059931 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:43.060025 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":293,"threads_started":1}
I20260811 12:12:43.511993 30336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:43.511180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:50667)
0811 12:12:43.511659 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:43.511798 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":263,"threads_started":1}
I20260811 12:12:43.967311 30338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:43.966588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:51701)
0811 12:12:43.967041 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:43.967120 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":246,"threads_started":1}
I20260811 12:12:44.062013 30339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:44.061305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.193:40917)
0811 12:12:44.061747 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:44.061836 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":208,"threads_started":1}
I20260811 12:12:44.513797 30340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:44.513087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.194:55603)
0811 12:12:44.513535 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:44.513627 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":227,"threads_started":1}
I20260811 12:12:44.956303 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 29549
I20260811 12:12:44.969085 30344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:44.968362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45431 (local address 127.4.48.195:39297)
0811 12:12:44.968794 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:44.968886 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:45431: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":216,"threads_started":1}
I20260811 12:12:44.971398 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:44.957290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.4.48.193:38611)
0811 12:12:44.957831 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:44.957839 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:44.957875 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:12:44.957955 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:44.957964 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:44.958207 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:44.958341 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:44.960521 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:44.961631 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:44.962814 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:44.963337 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:44.963430 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:44.965942 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:44.965954 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:44.965967 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:44.966005 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:44.968854 (+ 2849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:44.969612 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:44.969620 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:44.969622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:44.969763 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:44.970070 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:44.970073 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:44.970075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:44.970407 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:44.970634 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:44.970897 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 12:12:44.971086 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":261,"threads_started":1}
I20260811 12:12:44.972667 29794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:45.063838 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 29549
I20260811 12:12:45.087771 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 29805
I20260811 12:12:45.100833 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:45.088555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55152 (local address 127.4.48.194:39425)
0811 12:12:45.089116 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:45.089120 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:45.089136 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:12:45.089191 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:45.089194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:45.089253 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:45.089348 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:45.090783 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:45.091577 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:45.092859 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:45.093272 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:45.093385 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:45.095958 (+ 2573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:45.095963 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:45.095967 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:45.095998 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:45.098233 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:45.098881 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:45.098886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:45.098888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:45.099030 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:45.099402 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:45.099407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:45.099409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:45.099697 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:45.099980 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:45.100265 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 12:12:45.100485 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":334,"threads_started":1}
I20260811 12:12:45.101619 30016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:45.184900 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 29805
I20260811 12:12:45.216475 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 30031
I20260811 12:12:45.229604 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:45.217343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34018 (local address 127.4.48.195:42003)
0811 12:12:45.217935 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:45.217941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:45.217960 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:12:45.218016 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:45.218019 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:45.218085 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:45.218186 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:45.219703 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:45.220538 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:45.221859 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:45.222297 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:45.222413 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:45.224908 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:45.224913 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:45.224916 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:45.224952 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:45.227234 (+ 2282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:45.227931 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:45.227936 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:45.227938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:45.228081 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:45.228406 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:45.228409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:45.228411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:45.228644 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:45.228863 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:45.229049 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 12:12:45.229215 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":319,"threads_started":1}
I20260811 12:12:45.230261 30241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:45.312562 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 30031
2026-08-11T12:12:45Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24371 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:12:45 dist-test-slave-nj21 krb5kdc[30355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:12:45 dist-test-slave-nj21 krb5kdc[30355](info): set up 2 sockets
Aug 11 12:12:45 dist-test-slave-nj21 krb5kdc[30355](info): commencing operation
krb5kdc: starting...
W20260811 12:12:46.555884 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.197s user 0.012s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:12:46 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:12:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:12:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:12:46.815263 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:47.143009 30384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:47.143291 30384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:47.143437 30384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:47.150548 30384 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:12:47.150638 30384 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:12:47.150702 30384 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:12:47.150781 30384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:12:47.150853 30384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:47.150949 30384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:47.151005 30384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:12:47.151078 30384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:12:47.160620 30384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:47.161813 30384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:47.163084 30384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:47.174867 30395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:47.174943 30390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:47.174831 30391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:47.175329 30384 server_base.cc:1034] running on GCE node
I20260811 12:12:47.176373 30384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:47.178376 30384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:47.179602 30384 hybrid_clock.cc:648] HybridClock initialized: now 1786450367179542 us; error 33 us; skew 500 ppm
I20260811 12:12:47.179952 30384 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:47 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:47.182986 30384 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:12:47.184355 30384 webserver.cc:460] Webserver started at http://127.4.48.254:41837/ using document root <none> and password file <none>
I20260811 12:12:47.184998 30384 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:47.185083 30384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:47.185343 30384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:47.188285 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "ff34618a7b44452eaafc68f7b14587e9"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "166146d51d6c14275db74766bc20c4ee"
server_key_iv: "4106a5b0c2ad758347b8bdc5292c7b64"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.189028 30384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "ff34618a7b44452eaafc68f7b14587e9"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "166146d51d6c14275db74766bc20c4ee"
server_key_iv: "4106a5b0c2ad758347b8bdc5292c7b64"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.193949 30384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:12:47.197404 30407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:47.198701 30384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:12:47.198848 30384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "ff34618a7b44452eaafc68f7b14587e9"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "166146d51d6c14275db74766bc20c4ee"
server_key_iv: "4106a5b0c2ad758347b8bdc5292c7b64"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.199028 30384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:47.218418 30384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:47.222465 30384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:47.222674 30384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:47.240541 30384 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39031
I20260811 12:12:47.240552 30491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39031 every 8 connection(s)
I20260811 12:12:47.241921 30384 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:12:47.245671 30492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:47.247470 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 30384
I20260811 12:12:47.247692 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:12:47.248389 4291 external_mini_cluster.cc:1428] Setting key 3c4b6cff37463e0d779d6d4c960aeec4
I20260811 12:12:47.257470 30492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap starting.
I20260811 12:12:47.261166 30492 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:47.262476 30492 log.cc:824] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Log is configured to *not* fsync() on all Append() calls
Aug 11 12:12:47 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:47.265234 30492 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: No bootstrap required, opened a new log
I20260811 12:12:47.271126 30500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:47.250383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.4.48.254:39031)
0811 12:12:47.251194 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:47.251206 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:47.251252 (+ 46us) server_negotiation.cc:407] Connection header received
0811 12:12:47.252944 (+ 1692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:47.252967 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:47.253333 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:47.253653 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:47.255504 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.256848 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.258609 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.259262 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.259386 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:47.264184 (+ 4798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:47.264192 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:47.264207 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:47.264246 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:47.267161 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:47.267860 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:47.267867 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:47.267873 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:47.268053 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:47.268452 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:47.268456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:47.268458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:47.268975 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:47.269284 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:47.269764 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 12:12:47.270166 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":372,"threads_started":1}
I20260811 12:12:47.272763 30492 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:12:47.273074 30492 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:47.273169 30492 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Initialized, Role: FOLLOWER
I20260811 12:12:47.273677 30492 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:12:47.273947 30492 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:12:47.274086 30492 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:12:47.274224 30492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:47.276204 30492 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:12:47.276625 30492 leader_election.cc:302] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff34618a7b44452eaafc68f7b14587e9; no voters:
I20260811 12:12:47.277115 30492 leader_election.cc:288] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:12:47.277230 30503 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:47.277844 30503 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 LEADER]: Becoming Leader. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Running, Role: LEADER
I20260811 12:12:47.278324 30503 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:12:47.279232 30492 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:12:47.280998 30505 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff34618a7b44452eaafc68f7b14587e9. Latest consensus state: current_term: 1 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:12:47.280997 30504 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:12:47.281421 30504 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:47.281418 30505 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:12:47.285502 30510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:12:47.291109 30510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:12:47.299415 30510 catalog_manager.cc:1347] Generated new cluster ID: 27be41276b2d47b69c716df0f52cba73
I20260811 12:12:47.299537 30510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:12:47.317945 30510 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:12:47.319173 30510 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:12:47.325652 30510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Generated new TSK 0
I20260811 12:12:47.326393 30510 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:12:47.403436 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:47.757707 30535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:47.757974 30535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:47.758111 30535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:47.765174 30535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:47.765273 30535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:47.765430 30535 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:12:47.775044 30535 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:47.776345 30535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:47.777642 30535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:47.790072 30549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:47.790135 30544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:47.790113 30547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:47.790789 30535 server_base.cc:1034] running on GCE node
I20260811 12:12:47.791424 30535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:47.792361 30535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:47.793552 30535 hybrid_clock.cc:648] HybridClock initialized: now 1786450367793493 us; error 53 us; skew 500 ppm
I20260811 12:12:47.793826 30535 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:47 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:47.796934 30535 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:12:47.798266 30535 webserver.cc:460] Webserver started at http://127.4.48.193:41777/ using document root <none> and password file <none>
I20260811 12:12:47.798893 30535 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:47.799012 30535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:47.799304 30535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:47.802070 30535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "1ab21ab311290eda4aebcae9cde5d3e9"
server_key_iv: "b25e6158667173c26a3cf9866ffab25d"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.802839 30535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "1ab21ab311290eda4aebcae9cde5d3e9"
server_key_iv: "b25e6158667173c26a3cf9866ffab25d"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.807652 30535 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 12:12:47.810890 30557 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:47.812135 30535 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:12:47.812287 30535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "1ab21ab311290eda4aebcae9cde5d3e9"
server_key_iv: "b25e6158667173c26a3cf9866ffab25d"
server_key_version: "encryptionkey@0"
I20260811 12:12:47.812469 30535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:47.831729 30535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:47.835207 30535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:47.835460 30535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:47.836388 30535 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:47.837615 30535 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:47.837692 30535 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:47.837815 30535 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:47.837868 30535 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:47.867865 30535 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:42591
I20260811 12:12:47.867906 30729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:42591 every 8 connection(s)
I20260811 12:12:47.869045 30535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:12:47.874010 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 30535
I20260811 12:12:47.874233 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:12:47.875097 4291 external_mini_cluster.cc:1428] Setting key 309830993b0324f060c1e0c3e7cff9c3
Aug 11 12:12:47 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450367, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:47.889755 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:47.871456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:38453 (local address 127.4.48.254:39031)
0811 12:12:47.871957 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:47.871963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:47.873053 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 12:12:47.875444 (+ 2391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:47.875448 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:47.875520 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:47.875617 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:47.877131 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.877833 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.880026 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.880486 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.880614 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:47.884378 (+ 3764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:47.884383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:47.884386 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:47.884423 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:47.886784 (+ 2361us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:47.887480 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:47.887484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:47.887486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:47.887604 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:47.888038 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:47.888042 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:47.888044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:47.888300 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:47.888462 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:47.889164 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 12:12:47.889398 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":246,"threads_started":1}
I20260811 12:12:47.890645 30734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:47.871990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:38453)
0811 12:12:47.872884 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:47.872928 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:47.875155 (+ 2227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:47.875895 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:47.875902 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:47.876407 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:47.876964 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:47.876981 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.878011 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.878016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:47.879894 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:47.879909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:47.880736 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:47.880743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:47.881008 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:47.881661 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:47.881693 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:47.884160 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:47.886922 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:47.886931 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:47.886946 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:47.887343 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:47.887727 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:47.887732 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:47.887735 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:47.887918 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:47.888475 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:47.888483 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:47.888799 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 12:12:47.889028 (+ 229us) client_negotiation.cc:240] Negotiation successful
0811 12:12:47.889546 (+ 518us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"thread_start_us":391,"threads_started":1}
I20260811 12:12:47.892244 30730 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:12:47.892524 30730 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:47.893198 30730 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:12:47.895213 30440 ts_manager.cc:194] Registered new tserver with Master: ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591)
I20260811 12:12:47.896708 30440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:38453
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:12:47.936784 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:48.264391 30739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:48.264637 30739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:48.264741 30739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:48.271796 30739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:48.271893 30739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:48.272008 30739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:12:48.281371 30739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:48.282527 30739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:48.283804 30739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:48.296303 30750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:48.296271 30752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:48.296806 30748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:48.297557 30739 server_base.cc:1034] running on GCE node
I20260811 12:12:48.298159 30739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:48.299059 30739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:48.300259 30739 hybrid_clock.cc:648] HybridClock initialized: now 1786450368300207 us; error 35 us; skew 500 ppm
I20260811 12:12:48.300545 30739 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:48 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:48.303495 30739 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:12:48.304952 30739 webserver.cc:460] Webserver started at http://127.4.48.194:40845/ using document root <none> and password file <none>
I20260811 12:12:48.305605 30739 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:48.305732 30739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:48.306129 30739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:48.309070 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "0fa161aa35264cbcb182beabddd7b55e"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "2222de6eaea72af179fb37fed5ea8ef9"
server_key_iv: "f945078d46d520e576f810c60ce47fb3"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.309825 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "0fa161aa35264cbcb182beabddd7b55e"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "2222de6eaea72af179fb37fed5ea8ef9"
server_key_iv: "f945078d46d520e576f810c60ce47fb3"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.315032 30739 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:12:48.318492 30763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.319828 30739 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:12:48.319968 30739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "0fa161aa35264cbcb182beabddd7b55e"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "2222de6eaea72af179fb37fed5ea8ef9"
server_key_iv: "f945078d46d520e576f810c60ce47fb3"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.320127 30739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:48.333400 30739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:48.337013 30739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:48.337250 30739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:48.338106 30739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:48.339381 30739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:48.339452 30739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.339540 30739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:48.339586 30739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.370644 30739 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:34189
I20260811 12:12:48.370724 30961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:34189 every 8 connection(s)
I20260811 12:12:48.371958 30739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:12:48.380805 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 30739
I20260811 12:12:48.381006 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:12:48.381600 4291 external_mini_cluster.cc:1428] Setting key 0808f444848d00db53d11dd4ffc0a4d3
Aug 11 12:12:48 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:48.392606 30733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.374403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:36587 (local address 127.4.48.254:39031)
0811 12:12:48.374587 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.374591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.375748 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 12:12:48.377558 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.377564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.377649 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.377739 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:48.379252 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.380038 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.382214 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.382804 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.382935 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.387010 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:48.387015 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:48.387018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:48.387051 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:48.389556 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.390381 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.390385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.390387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.390504 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.390939 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.390942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.390944 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.391258 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:48.391402 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.392191 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.392356 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:12:48.393945 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.374843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:36587)
0811 12:12:48.375571 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:48.375615 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:48.377286 (+ 1671us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:48.377985 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:48.377992 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:48.378499 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:48.379061 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.379079 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.380196 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.380200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.382059 (+ 1859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.382074 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.383027 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.383034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.383278 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.384014 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:48.384045 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:48.386791 (+ 2746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:48.389707 (+ 2916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:48.389716 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:48.389730 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:48.390273 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:48.390603 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:48.390608 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:48.390612 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:48.390846 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:48.391392 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:48.391399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:48.391823 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 12:12:48.392124 (+ 301us) client_negotiation.cc:240] Negotiation successful
0811 12:12:48.392622 (+ 498us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":286,"threads_started":1}
I20260811 12:12:48.395335 30962 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:12:48.395653 30962 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:48.396436 30962 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:12:48.397986 30440 ts_manager.cc:194] Registered new tserver with Master: 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189)
I20260811 12:12:48.398767 30440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:36587
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:12:48.443042 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:12:48.774570 30969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:12:48.774833 30969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:12:48.774937 30969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:12:48.782051 30969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:12:48.782150 30969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:12:48.782298 30969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:12:48.791868 30969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:12:48.793040 30969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:12:48.794271 30969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:12:48.806769 30979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:48.806854 30982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:12:48.806821 30980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:12:48.807693 30969 server_base.cc:1034] running on GCE node
I20260811 12:12:48.808254 30969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:12:48.809089 30969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:12:48.810303 30969 hybrid_clock.cc:648] HybridClock initialized: now 1786450368810250 us; error 41 us; skew 500 ppm
I20260811 12:12:48.810597 30969 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:12:48 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:12:48.817359 30969 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:12:48.818765 30969 webserver.cc:460] Webserver started at http://127.4.48.195:42477/ using document root <none> and password file <none>
I20260811 12:12:48.819386 30969 fs_manager.cc:359] Metadata directory not provided
I20260811 12:12:48.819484 30969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:12:48.819793 30969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:12:48.822602 30969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "1a36f690ddb0592f14735fc09107ce8c"
server_key_iv: "6e386f9163f0f5d517da715eacf3f7c6"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.823299 30969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "1a36f690ddb0592f14735fc09107ce8c"
server_key_iv: "6e386f9163f0f5d517da715eacf3f7c6"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.828289 30969 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 12:12:48.831537 31002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.832821 30969 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:12:48.832962 30969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "1a36f690ddb0592f14735fc09107ce8c"
server_key_iv: "6e386f9163f0f5d517da715eacf3f7c6"
server_key_version: "encryptionkey@0"
I20260811 12:12:48.833112 30969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:12:48.859315 30969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:12:48.863176 30969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:12:48.863425 30969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:12:48.864395 30969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:12:48.865700 30969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:12:48.865772 30969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.865862 30969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:12:48.865904 30969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:12:48.896839 30969 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39259
I20260811 12:12:48.896895 31204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39259 every 8 connection(s)
I20260811 12:12:48.898128 30969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:12:48.899430 30730 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:12:48.907063 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 30969
I20260811 12:12:48.907241 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:12:48.907691 4291 external_mini_cluster.cc:1428] Setting key 301cdcbaf79a73053e5975eabb2de4a6
Aug 11 12:12:48 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450368, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:12:48.917575 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.900676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:33485 (local address 127.4.48.254:39031)
0811 12:12:48.901171 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.901176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.902425 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 12:12:48.904305 (+ 1880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.904310 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.904391 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.904495 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:48.905996 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.906870 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.908511 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.909068 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.909196 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.912663 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:48.912668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:48.912671 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:48.912702 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:48.914911 (+ 2209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.915619 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.915623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.915625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.915802 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.916162 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.916165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.916166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.916412 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:48.916542 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.917152 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.917324 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":246,"threads_started":1}
I20260811 12:12:48.918442 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.901278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:33485)
0811 12:12:48.902275 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:48.902312 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:48.904049 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:48.904736 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:48.904743 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:48.905252 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:12:48.905829 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.905848 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.907023 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.907027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.908363 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.908381 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.909282 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.909287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.909513 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.910156 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:12:48.910184 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:12:48.912473 (+ 2289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:12:48.915038 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:48.915045 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:48.915055 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:48.915518 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:48.915895 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:12:48.915898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:12:48.915900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:12:48.916078 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:12:48.916534 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:12:48.916539 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:12:48.916836 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 12:12:48.917049 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 12:12:48.917392 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":561,"threads_started":1}
I20260811 12:12:48.919700 31207 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:12:48.919982 31207 heartbeater.cc:464] Registering TS with master...
I20260811 12:12:48.920635 31207 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:12:48.922132 30440 ts_manager.cc:194] Registered new tserver with Master: b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:12:48.923076 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:12:48.923127 30440 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:33485
I20260811 12:12:48.939371 31209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.927381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.4.48.254:39031)
0811 12:12:48.927526 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.927532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.928001 (+ 469us) server_negotiation.cc:407] Connection header received
0811 12:12:48.928151 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.928156 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.928225 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.928306 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:48.929821 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.930500 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.931861 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.932260 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.932375 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.934595 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:48.934600 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:48.934603 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:48.934632 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:48.936830 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.937529 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.937533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.937535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.937645 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:48.938015 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:48.938019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:48.938023 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:48.938319 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:48.938572 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.938873 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.939079 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:12:48.943109 30440 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58346:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:12:48.949859 30440 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:12:48.982435 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.962074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38660 (local address 127.4.48.193:42591)
0811 12:12:48.974214 (+ 12140us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.974223 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.974255 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:12:48.974360 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.974370 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.974644 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.974851 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:48.975819 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.977417 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.980064 (+ 2647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.981313 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.981426 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.981659 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.981959 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.982122 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11937,"thread_start_us":11761,"threads_started":1}
I20260811 12:12:48.982540 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.961739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:38660)
0811 12:12:48.968961 (+ 7222us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:48.968999 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:48.969176 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:48.974901 (+ 5725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:48.974909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:48.974941 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:48.975595 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.975609 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.977570 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.977576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.979873 (+ 2297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.979887 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.981508 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.981515 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.981718 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.981770 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:12:48.981869 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:12:48.982048 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6980,"mutex_wait_us":59,"thread_start_us":6845,"threads_started":1}
I20260811 12:12:48.982575 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.961946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:34189 (local address 127.0.0.1:46328)
0811 12:12:48.973292 (+ 11346us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:48.973312 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:48.973442 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:48.974945 (+ 1503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:48.974949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:48.974967 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:48.975608 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.975618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.978254 (+ 2636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.978259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.980014 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.980027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.981630 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.981636 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.981815 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.981852 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:12:48.981930 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 12:12:48.982051 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11218,"mutex_wait_us":25,"thread_start_us":11573,"threads_started":1}
I20260811 12:12:48.983780 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.962412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46328 (local address 127.4.48.194:34189)
0811 12:12:48.974477 (+ 12065us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.974483 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.974507 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:12:48.974582 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.974587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.974750 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.975810 (+ 1060us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:48.976051 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.978108 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.980159 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.981385 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.983010 (+ 1625us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.983191 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.983268 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.983443 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11913,"thread_start_us":11785,"threads_started":1}
I20260811 12:12:48.984396 31233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.974685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.0.0.1:57946)
0811 12:12:48.975574 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:48.975593 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:48.975742 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:48.976689 (+ 947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:48.976693 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:48.976716 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:48.977093 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.977105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.978702 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.978708 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.980819 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:48.980833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.983284 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.983291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:48.983832 (+ 541us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.983868 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:12:48.983954 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 12:12:48.984066 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"thread_start_us":580,"threads_started":1}
I20260811 12:12:48.985111 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:48.969740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57946 (local address 127.4.48.195:39259)
0811 12:12:48.976266 (+ 6526us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:48.976272 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:48.976296 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:12:48.976368 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:48.976372 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:48.976515 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:48.976671 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:48.977296 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.978550 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.980997 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:48.982311 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:48.982448 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:48.983782 (+ 1334us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:48.984635 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 12:12:48.984789 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6389,"thread_start_us":6240,"threads_started":1}
I20260811 12:12:48.987056 30626 tablet_service.cc:1467] Processing CreateTablet for tablet bd88f43d530e40889906eaa892c87527 (DEFAULT_TABLE table=test-table [id=0888dad8bb584b1da886f28bea838983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:48.987075 30846 tablet_service.cc:1467] Processing CreateTablet for tablet bd88f43d530e40889906eaa892c87527 (DEFAULT_TABLE table=test-table [id=0888dad8bb584b1da886f28bea838983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:48.988361 30626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd88f43d530e40889906eaa892c87527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:48.988483 30846 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd88f43d530e40889906eaa892c87527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:48.988648 31094 tablet_service.cc:1467] Processing CreateTablet for tablet bd88f43d530e40889906eaa892c87527 (DEFAULT_TABLE table=test-table [id=0888dad8bb584b1da886f28bea838983]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:12:48.989982 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd88f43d530e40889906eaa892c87527. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:12:49.000105 31235 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Bootstrap starting.
I20260811 12:12:49.002451 31237 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Bootstrap starting.
I20260811 12:12:49.004860 31235 tablet_bootstrap.cc:654] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:49.005960 31237 tablet_bootstrap.cc:654] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:49.006290 31235 log.cc:824] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:49.007203 31237 log.cc:824] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:49.007807 31238 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Bootstrap starting.
I20260811 12:12:49.009545 31235 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: No bootstrap required, opened a new log
I20260811 12:12:49.009821 31235 ts_tablet_manager.cc:1397] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.006s
I20260811 12:12:49.011652 31238 tablet_bootstrap.cc:654] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Neither blocks nor log segments found. Creating new log.
I20260811 12:12:49.012791 31238 log.cc:824] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Log is configured to *not* fsync() on all Append() calls
I20260811 12:12:49.014204 31237 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: No bootstrap required, opened a new log
I20260811 12:12:49.014477 31237 ts_tablet_manager.cc:1397] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.005s
I20260811 12:12:49.015775 31238 tablet_bootstrap.cc:492] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: No bootstrap required, opened a new log
I20260811 12:12:49.016034 31238 ts_tablet_manager.cc:1397] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.003s
I20260811 12:12:49.017657 31235 raft_consensus.cc:348] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.017989 31235 raft_consensus.cc:374] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:49.018114 31235 raft_consensus.cc:729] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a1cbdd8dc4e16b5779225fcccff22, State: Initialized, Role: FOLLOWER
I20260811 12:12:49.018664 31235 consensus_queue.cc:260] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.024433 31235 ts_tablet_manager.cc:1428] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent starting tablet: real 0.014s user 0.014s sys 0.002s
I20260811 12:12:49.024617 31237 raft_consensus.cc:348] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.025090 31237 raft_consensus.cc:374] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:49.025192 31237 raft_consensus.cc:729] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa161aa35264cbcb182beabddd7b55e, State: Initialized, Role: FOLLOWER
I20260811 12:12:49.025897 31237 consensus_queue.cc:260] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.027011 30962 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:12:49.027611 31237 ts_tablet_manager.cc:1428] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Time spent starting tablet: real 0.013s user 0.006s sys 0.007s
I20260811 12:12:49.027701 31238 raft_consensus.cc:348] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.028707 31238 raft_consensus.cc:374] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:12:49.028818 31238 raft_consensus.cc:729] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16569f2c44a4d04a2a46aecd0e1d066, State: Initialized, Role: FOLLOWER
I20260811 12:12:49.029359 31238 consensus_queue.cc:260] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.030488 31207 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:12:49.031070 31238 ts_tablet_manager.cc:1428] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Time spent starting tablet: real 0.015s user 0.011s sys 0.001s
I20260811 12:12:49.076092 31242 raft_consensus.cc:482] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:12:49.076401 31242 raft_consensus.cc:504] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.078105 31242 leader_election.cc:288] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:12:49.086269 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.078675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:50247 (local address 127.4.48.194:34189)
0811 12:12:49.078883 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.078889 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.079731 (+ 842us) server_negotiation.cc:407] Connection header received
0811 12:12:49.080001 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.080007 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.080081 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.080163 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:49.080913 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.082026 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.084085 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.085315 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.085470 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.085544 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.085886 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.086016 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:12:49.086315 31252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.078508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:34189 (local address 127.4.48.193:50247)
0811 12:12:49.079619 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:49.079634 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:49.079815 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:49.080196 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:49.080200 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:49.080218 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:49.080717 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:49.080731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.082228 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.082234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:49.083934 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:49.083950 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.085542 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.085548 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:49.085695 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.085730 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:12:49.085829 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 12:12:49.085940 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1013,"mutex_wait_us":7,"spinlock_wait_cycles":22912,"thread_start_us":540,"threads_started":1}
I20260811 12:12:49.086403 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.078679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56563 (local address 127.4.48.195:39259)
0811 12:12:49.078870 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.078876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.079659 (+ 783us) server_negotiation.cc:407] Connection header received
0811 12:12:49.079931 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.079937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.080026 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.080127 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:12:49.080914 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.082026 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.084176 (+ 2150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.085318 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.085473 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.085544 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.086051 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.086183 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:12:49.087000 31253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.078441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.4.48.193:56563)
0811 12:12:49.079526 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 12:12:49.079549 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:12:49.079773 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:12:49.080157 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:12:49.080162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:12:49.080204 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:12:49.080786 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:49.080795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.082168 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.082174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:49.083962 (+ 1788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:12:49.083977 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.085719 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.085724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:12:49.085873 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.085905 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:12:49.086573 (+ 668us) client_negotiation.cc:240] Negotiation successful
0811 12:12:49.086697 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":926,"mutex_wait_us":72,"spinlock_wait_cycles":31488,"thread_start_us":791,"threads_started":1}
I20260811 12:12:49.087570 30880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd88f43d530e40889906eaa892c87527" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa161aa35264cbcb182beabddd7b55e" is_pre_election: true
I20260811 12:12:49.087965 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd88f43d530e40889906eaa892c87527" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" is_pre_election: true
I20260811 12:12:49.088094 30880 raft_consensus.cc:2454] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 0.
I20260811 12:12:49.088433 31125 raft_consensus.cc:2454] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 0.
I20260811 12:12:49.088754 30559 leader_election.cc:302] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa161aa35264cbcb182beabddd7b55e, ad8a1cbdd8dc4e16b5779225fcccff22; no voters:
I20260811 12:12:49.089144 31242 raft_consensus.cc:2781] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:12:49.089273 31242 raft_consensus.cc:482] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:12:49.089378 31242 raft_consensus.cc:3037] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:49.091540 31242 raft_consensus.cc:504] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.092196 31242 leader_election.cc:288] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:12:49.092532 30880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd88f43d530e40889906eaa892c87527" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa161aa35264cbcb182beabddd7b55e"
I20260811 12:12:49.092655 31125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd88f43d530e40889906eaa892c87527" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
I20260811 12:12:49.092715 30880 raft_consensus.cc:3037] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:49.092844 31125 raft_consensus.cc:3037] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:12:49.094892 30880 raft_consensus.cc:2454] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 1.
I20260811 12:12:49.094964 31125 raft_consensus.cc:2454] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 1.
I20260811 12:12:49.095230 30559 leader_election.cc:302] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa161aa35264cbcb182beabddd7b55e, ad8a1cbdd8dc4e16b5779225fcccff22; no voters:
I20260811 12:12:49.095490 31242 raft_consensus.cc:2781] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:12:49.095947 31242 raft_consensus.cc:686] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 LEADER]: Becoming Leader. State: Replica: ad8a1cbdd8dc4e16b5779225fcccff22, State: Running, Role: LEADER
I20260811 12:12:49.096391 31242 consensus_queue.cc:237] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:12:49.100723 30439 catalog_manager.cc:5495] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } health_report { overall_health: UNKNOWN } } }
W20260811 12:12:49.121241 30732 tablet.cc:2369] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:12:49.123910 30963 tablet.cc:2369] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:12:49.124191 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.118615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38674 (local address 127.4.48.193:42591)
0811 12:12:49.118829 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.118833 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.118858 (+ 25us) server_negotiation.cc:407] Connection header received
0811 12:12:49.119116 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.119121 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.119204 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.119333 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:12:49.119715 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.120435 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.121964 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.122387 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.122502 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.123022 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:12:49.123542 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:12:49.123742 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.123845 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.123928 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:12:49.132288 30880 raft_consensus.cc:1261] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Refusing update from remote peer ad8a1cbdd8dc4e16b5779225fcccff22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:49.132290 31125 raft_consensus.cc:1261] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Refusing update from remote peer ad8a1cbdd8dc4e16b5779225fcccff22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:12:49.133193 31254 consensus_queue.cc:1035] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:49.133571 31242 consensus_queue.cc:1035] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:12:49.145080 31258 mvcc.cc:205] Tried to move back new op lower bound from 7317300711955746816 to 7317300711822381056. Current Snapshot: MvccSnapshot[applied={T|T < 7317300711955746816}]
I20260811 12:12:49.157188 30440 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:49.157461 30440 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58346:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:12:49.161000 30440 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Sending DeleteTablet for 3 replicas of tablet bd88f43d530e40889906eaa892c87527
I20260811 12:12:49.162212 30846 tablet_service.cc:1514] Processing DeleteTablet for tablet bd88f43d530e40889906eaa892c87527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:49 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:46328
I20260811 12:12:49.162217 30626 tablet_service.cc:1514] Processing DeleteTablet for tablet bd88f43d530e40889906eaa892c87527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:49 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:38660
I20260811 12:12:49.162315 31094 tablet_service.cc:1514] Processing DeleteTablet for tablet bd88f43d530e40889906eaa892c87527 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:12:49 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:57946
I20260811 12:12:49.163537 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 30535
I20260811 12:12:49.172290 31279 tablet_replica.cc:330] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: stopping tablet replica
I20260811 12:12:49.173243 31279 raft_consensus.cc:2229] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:49.173883 31279 raft_consensus.cc:2258] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:49.176155 31278 tablet_replica.cc:330] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: stopping tablet replica
I20260811 12:12:49.176785 31279 ts_tablet_manager.cc:1905] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:49.177266 31278 raft_consensus.cc:2229] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:12:49.179342 31278 raft_consensus.cc:2258] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:49.181640 31278 ts_tablet_manager.cc:1905] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:12:49.187196 31279 ts_tablet_manager.cc:1918] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:49.187856 31277 tablet_replica.cc:330] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: stopping tablet replica
I20260811 12:12:49.188330 31277 raft_consensus.cc:2229] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:12:49.189087 31277 raft_consensus.cc:2258] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:12:49.191375 31277 ts_tablet_manager.cc:1905] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:12:49 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:12:49.194383 31278 ts_tablet_manager.cc:1918] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:49.194583 31278 log.cc:1196] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/bd88f43d530e40889906eaa892c87527
I20260811 12:12:49.195153 31278 ts_tablet_manager.cc:1939] T bd88f43d530e40889906eaa892c87527 P 0fa161aa35264cbcb182beabddd7b55e: Deleting consensus metadata
I20260811 12:12:49.197284 30410 catalog_manager.cc:4841] TS 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): tablet bd88f43d530e40889906eaa892c87527 (table test-table [id=0888dad8bb584b1da886f28bea838983]) successfully deleted
I20260811 12:12:49.197844 31279 log.cc:1196] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/bd88f43d530e40889906eaa892c87527
I20260811 12:12:49.198563 31279 ts_tablet_manager.cc:1939] T bd88f43d530e40889906eaa892c87527 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting consensus metadata
I20260811 12:12:49.198788 31277 ts_tablet_manager.cc:1918] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:12:49.199030 31277 log.cc:1196] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/bd88f43d530e40889906eaa892c87527
I20260811 12:12:49.199592 31277 ts_tablet_manager.cc:1939] T bd88f43d530e40889906eaa892c87527 P ad8a1cbdd8dc4e16b5779225fcccff22: Deleting consensus metadata
I20260811 12:12:49.200472 30409 catalog_manager.cc:4841] TS b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): tablet bd88f43d530e40889906eaa892c87527 (table test-table [id=0888dad8bb584b1da886f28bea838983]) successfully deleted
I20260811 12:12:49.201527 30409 catalog_manager.cc:4841] TS ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591): tablet bd88f43d530e40889906eaa892c87527 (table test-table [id=0888dad8bb584b1da886f28bea838983]) successfully deleted
I20260811 12:12:49.202491 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.164759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38682 (local address 127.4.48.193:42591)
0811 12:12:49.164942 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.164948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.180313 (+ 15365us) server_negotiation.cc:407] Connection header received
0811 12:12:49.180844 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.180850 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.180940 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.181026 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:49.183381 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.184814 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.187666 (+ 2852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.188377 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.188498 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.194415 (+ 5917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:49.194432 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:49.194462 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:49.194527 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:49.199124 (+ 4597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.200102 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.200111 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.200114 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.200324 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.200865 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.200869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.200871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.201289 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:49.201585 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.202017 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.202209 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:12:49.203248 30725 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:49.300904 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 30535
I20260811 12:12:49.325922 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 30739
Aug 11 12:12:49 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:12:49.340500 31229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.326747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46342 (local address 127.4.48.194:34189)
0811 12:12:49.326928 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.326932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.326951 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:12:49.327010 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.327014 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.327083 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.327161 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:49.328484 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.329234 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.330451 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.330818 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.330931 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.335106 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:49.335120 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:49.335137 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:49.335172 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:49.337907 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.338540 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.338547 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.338550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.338673 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.338969 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.338972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.338974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.339349 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:49.339573 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.339952 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.340161 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:12:49.341180 30953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:49.430222 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 30739
I20260811 12:12:49.453657 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 30969
Aug 11 12:12:49 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450366, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:12:49.467741 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:12:49.454467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57956 (local address 127.4.48.195:39259)
0811 12:12:49.454647 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 12:12:49.454651 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:12:49.454672 (+ 21us) server_negotiation.cc:407] Connection header received
0811 12:12:49.454776 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:12:49.454780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:12:49.454844 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:12:49.454924 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:12:49.456247 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.456972 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.458187 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:12:49.458589 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:12:49.458703 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:12:49.462778 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:12:49.462795 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:12:49.462815 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:12:49.462848 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:12:49.465349 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.465977 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.465984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.465986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.466118 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:12:49.466402 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:12:49.466404 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:12:49.466406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:12:49.466754 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:12:49.466963 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 12:12:49.467215 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 12:12:49.467375 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 12:12:49.468353 31192 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:49.554867 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 30969
I20260811 12:12:49.577299 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 30384
I20260811 12:12:49.578549 30483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:12:49.690135 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 30384
I20260811 12:13:09.718405 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--webserver_interface=127.4.48.254
--webserver_port=41837
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:10.067513 31289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:10.067804 31289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:10.067931 31289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:10.074950 31289 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:13:10.075044 31289 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:13:10.075109 31289 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:13:10.075178 31289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:13:10.075258 31289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:10.075354 31289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:10.075414 31289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:13:10.075496 31289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:13:10.085009 31289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=41837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:10.086236 31289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:10.087535 31289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:10.111292 31306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:10.111316 31301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:10.111348 31302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:10.112269 31289 server_base.cc:1034] running on GCE node
I20260811 12:13:10.113135 31289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:10.114638 31289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:10.115849 31289 hybrid_clock.cc:648] HybridClock initialized: now 1786450390115786 us; error 34 us; skew 500 ppm
I20260811 12:13:10.116227 31289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:10 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450390, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:10.122880 31289 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:13:10.124575 31289 webserver.cc:460] Webserver started at http://127.4.48.254:41837/ using document root <none> and password file <none>
I20260811 12:13:10.125379 31289 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:10.125512 31289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:10.131485 31289 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:13:10.134555 31315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:10.135999 31289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:13:10.136176 31289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "ff34618a7b44452eaafc68f7b14587e9"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "166146d51d6c14275db74766bc20c4ee"
server_key_iv: "4106a5b0c2ad758347b8bdc5292c7b64"
server_key_version: "encryptionkey@0"
I20260811 12:13:10.137252 31289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:10.161924 31289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:10.166610 31289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:10.166916 31289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:10.184899 31289 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39031
I20260811 12:13:10.184942 31390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39031 every 8 connection(s)
I20260811 12:13:10.186415 31289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:13:10.190110 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 31289
I20260811 12:13:10.191128 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:42591
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=41777
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:13:10.195034 31392 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:13:10.199481 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap starting.
I20260811 12:13:10.220515 31392 log.cc:824] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:10.233083 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:13:10.233590 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap complete.
I20260811 12:13:10.243468 31392 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:10.244359 31392 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Initialized, Role: FOLLOWER
I20260811 12:13:10.245055 31392 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:10.245294 31392 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:13:10.245409 31392 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:13:10.245573 31392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 1 FOLLOWER]: Advancing to term 2
I20260811 12:13:10.249397 31392 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:10.249929 31392 leader_election.cc:302] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff34618a7b44452eaafc68f7b14587e9; no voters:
I20260811 12:13:10.250591 31392 leader_election.cc:288] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 12:13:10.250725 31398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 12:13:10.252310 31398 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 LEADER]: Becoming Leader. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Running, Role: LEADER
I20260811 12:13:10.252925 31398 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:10.253578 31392 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:13:10.255553 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff34618a7b44452eaafc68f7b14587e9. Latest consensus state: current_term: 2 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:13:10.255600 31399 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:13:10.255976 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:10.256026 31399 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:10.262239 31406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:13:10.271097 31406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:13:10.272656 31406 catalog_manager.cc:1259] Loaded cluster ID: 27be41276b2d47b69c716df0f52cba73
I20260811 12:13:10.272810 31406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:13:10.277132 31406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:13:10.278798 31406 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Loaded TSK: 0
I20260811 12:13:10.297004 31427 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:13:10.306143 31406 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Generated new TSK 1
I20260811 12:13:10.306483 31406 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:13:10.571331 31394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:10.571602 31394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:10.571743 31394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:10.578761 31394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:10.578862 31394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:10.578999 31394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:13:10.588493 31394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:42591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=41777
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:10.589772 31394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:10.591082 31394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:10.605417 31436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:10.605732 31435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:10.606110 31439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:10.606582 31394 server_base.cc:1034] running on GCE node
I20260811 12:13:10.607173 31394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:10.608166 31394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:10.609385 31394 hybrid_clock.cc:648] HybridClock initialized: now 1786450390609336 us; error 41 us; skew 500 ppm
I20260811 12:13:10.609697 31394 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:10 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450390, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:10.612915 31394 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:13:10.614454 31394 webserver.cc:460] Webserver started at http://127.4.48.193:41777/ using document root <none> and password file <none>
I20260811 12:13:10.615113 31394 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:10.615242 31394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:10.620788 31394 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 12:13:10.624096 31450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:10.625721 31394 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:13:10.625901 31394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "1ab21ab311290eda4aebcae9cde5d3e9"
server_key_iv: "b25e6158667173c26a3cf9866ffab25d"
server_key_version: "encryptionkey@0"
I20260811 12:13:10.627208 31394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:10.650507 31394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:10.655405 31394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:10.655702 31394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:10.656857 31394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:10.658437 31394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:10.658543 31394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:10.658646 31394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:10.658716 31394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:10.689915 31394 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:42591
I20260811 12:13:10.689982 31628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:42591 every 8 connection(s)
I20260811 12:13:10.691336 31394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:13:10.701792 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 31394
I20260811 12:13:10.702577 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:34189
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=40845
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:13:10 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450390, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:10.713863 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:10.694212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:40375)
0811 12:13:10.694986 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:10.695024 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:10.696690 (+ 1666us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:10.697912 (+ 1222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:10.697918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:10.698437 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:10.698946 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:10.698963 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:10.700589 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:10.700593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:10.701774 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:10.701786 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:10.703126 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:10.703131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:10.703346 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:10.703986 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:10.704008 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:10.706437 (+ 2429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:10.709620 (+ 3183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:10.709629 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:10.709643 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:10.710038 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:10.710465 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:10.710470 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:10.710474 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:10.710664 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:10.711369 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:10.711377 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:10.711738 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 12:13:10.711995 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 12:13:10.712536 (+ 541us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":333,"threads_started":1}
I20260811 12:13:10.713863 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:10.694132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:40375 (local address 127.4.48.254:39031)
0811 12:13:10.694783 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:10.694800 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:10.695144 (+ 344us) server_negotiation.cc:407] Connection header received
0811 12:13:10.697183 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:10.697195 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:10.697515 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:10.697787 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:10.699101 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:10.700421 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:10.701934 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:10.702909 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:10.703088 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:10.706690 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:10.706698 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:10.706712 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:10.706768 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:10.709481 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:10.710170 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:10.710177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:10.710182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:10.710354 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:10.710780 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:10.710784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:10.710786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:10.711229 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:10.711432 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:10.712214 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 12:13:10.712599 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":246,"threads_started":1}
I20260811 12:13:10.717406 31629 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:13:10.717687 31629 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:10.718394 31629 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:13:10.720925 31339 ts_manager.cc:194] Registered new tserver with Master: ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591)
I20260811 12:13:10.722927 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:40375
W20260811 12:13:11.038815 31637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:11.039088 31637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:11.039183 31637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:11.046168 31637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:11.046271 31637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:11.046387 31637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:13:11.055826 31637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:34189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=40845
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:11.057039 31637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:11.058332 31637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:11.071219 31644 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:11.071127 31651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:11.071161 31646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:11.071979 31637 server_base.cc:1034] running on GCE node
I20260811 12:13:11.072533 31637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:11.073433 31637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:11.074617 31637 hybrid_clock.cc:648] HybridClock initialized: now 1786450391074565 us; error 47 us; skew 500 ppm
I20260811 12:13:11.074910 31637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:11 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:11.077963 31637 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:13:11.079304 31637 webserver.cc:460] Webserver started at http://127.4.48.194:40845/ using document root <none> and password file <none>
I20260811 12:13:11.080013 31637 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:11.080133 31637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:11.085263 31637 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:13:11.087837 31660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.089066 31637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:11.089212 31637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "0fa161aa35264cbcb182beabddd7b55e"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "2222de6eaea72af179fb37fed5ea8ef9"
server_key_iv: "f945078d46d520e576f810c60ce47fb3"
server_key_version: "encryptionkey@0"
I20260811 12:13:11.090169 31637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:11.120311 31637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:11.124634 31637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:11.124888 31637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:11.125907 31637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:11.127231 31637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:11.127329 31637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.127441 31637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:11.127513 31637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.160964 31637 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:34189
I20260811 12:13:11.161020 31836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:34189 every 8 connection(s)
I20260811 12:13:11.162397 31637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:13:11.164786 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 31637
I20260811 12:13:11.165417 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:39259
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=42477
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 12:13:11 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:11.184139 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.165034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:36609 (local address 127.4.48.254:39031)
0811 12:13:11.165252 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.165258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.166572 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 12:13:11.168905 (+ 2333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.168910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.168988 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.169073 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:11.170973 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.171778 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.173815 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.174227 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.174318 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.178135 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:11.178141 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:11.178146 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:11.178186 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:11.181173 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.181869 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.181874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.181877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.182021 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.182393 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.182396 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.182399 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.182784 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:11.182924 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.183527 (+ 603us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.183773 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:13:11.185065 31841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.165532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:36609)
0811 12:13:11.166378 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.166444 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.168633 (+ 2189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.169329 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.169339 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.170038 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:11.170791 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.170814 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.171941 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.171946 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.173694 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.173708 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.174450 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.174457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.174712 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.175375 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:11.175401 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:11.177896 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:11.181307 (+ 3411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:11.181319 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:11.181330 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:11.181748 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:11.182118 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:11.182121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:11.182123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:11.182293 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:11.182894 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:11.182899 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:11.183183 (+ 284us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.183398 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.183812 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":292,"threads_started":1}
I20260811 12:13:11.186657 31838 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:13:11.186980 31838 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:11.187849 31838 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:13:11.189558 31339 ts_manager.cc:194] Registered new tserver with Master: 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189)
I20260811 12:13:11.190716 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:36609
W20260811 12:13:11.503494 31843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:11.503786 31843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:11.503926 31843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:11.511157 31843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:11.511268 31843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:11.511415 31843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:13:11.521073 31843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:39259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=42477
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:11.522336 31843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:11.523620 31843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:11.536466 31859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:11.536382 31861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:11.536432 31867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:11.537318 31843 server_base.cc:1034] running on GCE node
I20260811 12:13:11.537876 31843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:11.538731 31843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:11.539918 31843 hybrid_clock.cc:648] HybridClock initialized: now 1786450391539864 us; error 40 us; skew 500 ppm
I20260811 12:13:11.540197 31843 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:11 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:11.543136 31843 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:13:11.544553 31843 webserver.cc:460] Webserver started at http://127.4.48.195:42477/ using document root <none> and password file <none>
I20260811 12:13:11.545169 31843 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:11.545284 31843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:11.550311 31843 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 12:13:11.552860 31877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.554104 31843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:11.554261 31843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "1a36f690ddb0592f14735fc09107ce8c"
server_key_iv: "6e386f9163f0f5d517da715eacf3f7c6"
server_key_version: "encryptionkey@0"
I20260811 12:13:11.555169 31843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:11.574132 31843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:11.578006 31843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:11.578240 31843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:11.579135 31843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:11.580529 31843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:11.580634 31843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.580745 31843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:11.580816 31843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:11.611156 31843 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39259
I20260811 12:13:11.611200 32063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39259 every 8 connection(s)
I20260811 12:13:11.612527 31843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:13:11.618204 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 31843
Aug 11 12:13:11 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450391, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:11.632332 31633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.614987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:41673 (local address 127.4.48.254:39031)
0811 12:13:11.615172 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.615176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.616292 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 12:13:11.618054 (+ 1762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.618060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.618137 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.618223 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:11.619927 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.620660 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.622143 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.622681 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.622798 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.626318 (+ 3520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:11.626323 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:11.626326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:11.626357 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:11.629058 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.629907 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.629914 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.629917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.630068 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.630547 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.630550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.630552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.630807 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:11.630914 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.631766 (+ 852us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.631965 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":65}
I20260811 12:13:11.633513 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.619061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45874 (local address 127.4.48.254:39031)
0811 12:13:11.619539 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.619544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.619918 (+ 374us) server_negotiation.cc:407] Connection header received
0811 12:13:11.620096 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.620101 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.620157 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.620242 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:11.621903 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.622708 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.624166 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.624626 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.624731 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.627730 (+ 2999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:11.627737 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:11.627742 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:11.627786 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:11.630676 (+ 2890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.631431 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.631442 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.631444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.631549 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.631961 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.631965 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.631969 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.632270 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:11.632551 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.633081 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.633231 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":370,"thread_start_us":237,"threads_started":1}
I20260811 12:13:11.633616 32068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.615391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:41673)
0811 12:13:11.616128 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.616164 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.617791 (+ 1627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.618481 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.618489 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.619046 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:11.619764 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.619785 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.620823 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.620827 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.622004 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.622017 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.622899 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.622907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.623147 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.623763 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:11.623785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:11.626114 (+ 2329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:11.629186 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:11.629193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:11.629204 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:11.629781 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:11.630176 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:11.630181 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:11.630185 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:11.630433 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:11.630929 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:11.630937 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:11.631338 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.631644 (+ 306us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.632135 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":328,"threads_started":1}
I20260811 12:13:11.634922 32066 heartbeater.cc:347] Connected to a master server at 127.4.48.254:39031
I20260811 12:13:11.635215 32066 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:11.635970 32066 heartbeater.cc:511] Master 127.4.48.254:39031 requested a full tablet report, sending...
I20260811 12:13:11.637355 31339 ts_manager.cc:194] Registered new tserver with Master: b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:13:11.638144 31339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:41673
I20260811 12:13:11.648672 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:13:11.655465 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.649740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45882 (local address 127.4.48.254:39031)
0811 12:13:11.649936 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.649942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.650275 (+ 333us) server_negotiation.cc:407] Connection header received
0811 12:13:11.650495 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.650499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.650624 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.650746 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:13:11.651181 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.651995 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.653530 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.653964 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.654074 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.654568 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:13:11.654831 (+ 263us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 12:13:11.655175 (+ 344us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45882: token expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 12:13:11.655565 32069 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45882: token expired
W20260811 12:13:11.656152 31215 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.4.48.254:39031)
I20260811 12:13:11.668155 32069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.656719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.4.48.254:39031)
0811 12:13:11.656891 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.656897 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.656912 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:13:11.656953 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.656957 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.657005 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.657080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:11.658523 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.659193 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.660743 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.661117 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.661204 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.663662 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:11.663687 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:11.663692 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:11.663728 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:11.666000 (+ 2272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.666670 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.666673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.666675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.666789 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.667062 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.667064 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.667066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.667339 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:11.667520 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.667777 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.667933 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:13:11.671293 31339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:13:11.678515 31339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:13:11.703169 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.694983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.4.48.193:42591)
0811 12:13:11.695537 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.695543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.695565 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:13:11.695641 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.695646 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.695907 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.696124 (+ 217us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.696829 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.698325 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.700883 (+ 2558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.702041 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.702165 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.702397 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.702704 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.702859 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":292,"threads_started":1}
I20260811 12:13:11.703228 32081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.690499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36216)
0811 12:13:11.691288 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.691334 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.691497 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.696156 (+ 4659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.696165 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.696219 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.696606 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.696622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.698458 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.698463 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.700678 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.700693 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.702267 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.702273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.702480 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.702517 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.702646 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.702828 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":17,"spinlock_wait_cycles":3328,"thread_start_us":5320,"threads_started":1}
I20260811 12:13:11.704550 32079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.690619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:34189 (local address 127.0.0.1:58666)
0811 12:13:11.696078 (+ 5459us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.696096 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.696214 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.697031 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.697035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.697060 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.697507 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.697524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.699302 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.699309 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.701432 (+ 2123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.701451 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.703714 (+ 2263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.703721 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.703883 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.703916 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.704007 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.704137 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5345,"spinlock_wait_cycles":2816,"thread_start_us":324,"threads_started":1}
I20260811 12:13:11.705377 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.691027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58666 (local address 127.4.48.194:34189)
0811 12:13:11.692407 (+ 1380us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.692417 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.696342 (+ 3925us) server_negotiation.cc:407] Connection header received
0811 12:13:11.696436 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.696441 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.696589 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.696767 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.697720 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.699155 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.701608 (+ 2453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.702997 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.703121 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.704857 (+ 1736us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.704923 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.705094 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1225,"thread_start_us":1132,"threads_started":1}
I20260811 12:13:11.705582 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.696449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37444 (local address 127.4.48.195:39259)
0811 12:13:11.697079 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.697087 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.697744 (+ 657us) server_negotiation.cc:407] Connection header received
0811 12:13:11.698054 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.698062 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.698255 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.698400 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.699373 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.700553 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.702892 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.704258 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.704351 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.704567 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.705136 (+ 569us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.705289 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":460,"thread_start_us":292,"threads_started":1}
I20260811 12:13:11.705503 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.697053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.0.0.1:37444)
0811 12:13:11.697643 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.697665 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.697845 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.698690 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.698695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.698721 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.699163 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.699177 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.700710 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.700715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.702706 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.702722 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.704766 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.704772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.704933 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.704964 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.705054 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.705164 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":24,"thread_start_us":299,"threads_started":1}
I20260811 12:13:11.706982 31527 tablet_service.cc:1467] Processing CreateTablet for tablet 79949417ed2b4efc9e797438506fe2c2 (DEFAULT_TABLE table=test-table [id=ab5ceca9df524a4e9dd460fdff3d0687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:11.708696 31727 tablet_service.cc:1467] Processing CreateTablet for tablet 79949417ed2b4efc9e797438506fe2c2 (DEFAULT_TABLE table=test-table [id=ab5ceca9df524a4e9dd460fdff3d0687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:11.709197 31527 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79949417ed2b4efc9e797438506fe2c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:11.709790 31727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79949417ed2b4efc9e797438506fe2c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:11.710954 31949 tablet_service.cc:1467] Processing CreateTablet for tablet 79949417ed2b4efc9e797438506fe2c2 (DEFAULT_TABLE table=test-table [id=ab5ceca9df524a4e9dd460fdff3d0687]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:11.712445 31949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79949417ed2b4efc9e797438506fe2c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:11.720808 32091 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Bootstrap starting.
I20260811 12:13:11.724367 32091 tablet_bootstrap.cc:654] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:11.725369 32091 log.cc:824] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:11.726298 31629 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:13:11.727058 32092 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Bootstrap starting.
I20260811 12:13:11.728603 32091 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: No bootstrap required, opened a new log
I20260811 12:13:11.729481 32091 ts_tablet_manager.cc:1397] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:13:11.730525 32092 tablet_bootstrap.cc:654] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:11.731789 32092 log.cc:824] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:11.733328 32096 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Bootstrap starting.
I20260811 12:13:11.735073 32092 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: No bootstrap required, opened a new log
I20260811 12:13:11.735327 32092 ts_tablet_manager.cc:1397] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 12:13:11.736649 32096 tablet_bootstrap.cc:654] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:11.737896 32096 log.cc:824] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:11.740443 32091 raft_consensus.cc:348] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.740876 32091 raft_consensus.cc:374] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:11.740978 32091 raft_consensus.cc:729] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa161aa35264cbcb182beabddd7b55e, State: Initialized, Role: FOLLOWER
I20260811 12:13:11.741598 32091 consensus_queue.cc:260] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.742400 32096 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: No bootstrap required, opened a new log
I20260811 12:13:11.742658 32096 ts_tablet_manager.cc:1397] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260811 12:13:11.742777 31838 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:13:11.744289 32092 raft_consensus.cc:348] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.744660 32091 ts_tablet_manager.cc:1428] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Time spent starting tablet: real 0.015s user 0.004s sys 0.008s
I20260811 12:13:11.744727 32092 raft_consensus.cc:374] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:11.744849 32092 raft_consensus.cc:729] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a1cbdd8dc4e16b5779225fcccff22, State: Initialized, Role: FOLLOWER
I20260811 12:13:11.745458 32092 consensus_queue.cc:260] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.747210 32092 ts_tablet_manager.cc:1428] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:13:11.751317 32096 raft_consensus.cc:348] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.751612 32096 raft_consensus.cc:374] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:11.751726 32096 raft_consensus.cc:729] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b16569f2c44a4d04a2a46aecd0e1d066, State: Initialized, Role: FOLLOWER
I20260811 12:13:11.752241 32096 consensus_queue.cc:260] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.753171 32066 heartbeater.cc:503] Master 127.4.48.254:39031 was elected leader, sending a full tablet report...
I20260811 12:13:11.753564 32096 ts_tablet_manager.cc:1428] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 12:13:11.782685 32104 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:11.782948 32104 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.783866 32103 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:11.784166 32103 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.784970 32104 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:13:11.785969 32103 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:13:11.793820 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.785764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37639 (local address 127.4.48.194:34189)
0811 12:13:11.785971 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.785977 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.786220 (+ 243us) server_negotiation.cc:407] Connection header received
0811 12:13:11.786557 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.786563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.786640 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.786706 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.787530 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.788648 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.791428 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.792641 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.792751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.792825 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.793413 (+ 588us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.793539 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:13:11.794224 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.785394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:56751 (local address 127.4.48.195:39259)
0811 12:13:11.785598 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.785604 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.786713 (+ 1109us) server_negotiation.cc:407] Connection header received
0811 12:13:11.786988 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.786993 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.787065 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.787128 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.788006 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.789068 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.791729 (+ 2661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.793008 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.793139 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.793210 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.793824 (+ 614us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.793975 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 12:13:11.794468 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.786368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:54361 (local address 127.4.48.193:42591)
0811 12:13:11.786556 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.786561 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.787340 (+ 779us) server_negotiation.cc:407] Connection header received
0811 12:13:11.787574 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.787579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.787649 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.787749 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.788523 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.789737 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.792204 (+ 2467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.793336 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.793473 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.793539 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.794111 (+ 572us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.794220 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:13:11.794661 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.785151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:34189 (local address 127.4.48.193:37639)
0811 12:13:11.786118 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.786139 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.786350 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.786929 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.786933 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.786961 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.787351 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.787364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.788819 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.788824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.791233 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.791249 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.793033 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.793040 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.793219 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.793252 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.794081 (+ 829us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.794211 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":96,"thread_start_us":618,"threads_started":1}
I20260811 12:13:11.795186 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.786385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.4.48.194:54361)
0811 12:13:11.787241 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.787260 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.787434 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.787944 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.787949 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.787978 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.788368 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.788378 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.789865 (+ 1487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.789870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.792022 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.792037 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.793744 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.793749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.793928 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.793959 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.794706 (+ 747us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.794834 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":502,"threads_started":1}
I20260811 12:13:11.795616 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa161aa35264cbcb182beabddd7b55e" is_pre_election: true
I20260811 12:13:11.795734 32108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.785311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.4.48.193:56751)
0811 12:13:11.786595 (+ 1284us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.786615 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.786835 (+ 220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.787335 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.787339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.787364 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.787803 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.787816 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.789248 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.789254 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.791519 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.791534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.793456 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.793462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.793632 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.793663 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.795066 (+ 1403us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.795206 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1157,"mutex_wait_us":1,"thread_start_us":1245,"threads_started":1}
I20260811 12:13:11.796149 31758 raft_consensus.cc:2454] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 0.
I20260811 12:13:11.797551 31556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "0fa161aa35264cbcb182beabddd7b55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" is_pre_election: true
I20260811 12:13:11.797667 31982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" is_pre_election: true
I20260811 12:13:11.797950 31556 raft_consensus.cc:2454] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fa161aa35264cbcb182beabddd7b55e in term 0.
I20260811 12:13:11.798198 31452 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa161aa35264cbcb182beabddd7b55e, ad8a1cbdd8dc4e16b5779225fcccff22; no voters:
I20260811 12:13:11.798223 31982 raft_consensus.cc:2454] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 0.
I20260811 12:13:11.798629 32104 raft_consensus.cc:2781] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:13:11.798776 32104 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:13:11.798872 32104 raft_consensus.cc:3037] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:11.798807 31661 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fa161aa35264cbcb182beabddd7b55e, ad8a1cbdd8dc4e16b5779225fcccff22; no voters:
I20260811 12:13:11.799331 32103 raft_consensus.cc:2781] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:13:11.799467 32103 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:13:11.799567 32103 raft_consensus.cc:3037] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:11.801541 32104 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.801873 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.787774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:59367 (local address 127.4.48.195:39259)
0811 12:13:11.794457 (+ 6683us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.794463 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.795503 (+ 1040us) server_negotiation.cc:407] Connection header received
0811 12:13:11.795762 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.795768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.795847 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.795906 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:11.796423 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.797534 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.799632 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.800816 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.800963 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.801037 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.801501 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.801619 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6578,"thread_start_us":8718,"threads_started":1}
I20260811 12:13:11.802075 32112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.787320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.4.48.194:59367)
0811 12:13:11.795409 (+ 8089us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:11.795426 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:11.795584 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:11.796074 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:11.796078 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:11.796101 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:11.796262 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.796272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.797708 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.797713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.799436 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:11.799457 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.801109 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.801115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:11.801320 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.801352 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:13:11.801459 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 12:13:11.801595 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7978,"thread_start_us":9328,"threads_started":1}
I20260811 12:13:11.802518 32103 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.802651 32104 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 election: Requested vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:13:11.802668 31758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fa161aa35264cbcb182beabddd7b55e"
I20260811 12:13:11.802758 31982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
I20260811 12:13:11.802943 31758 raft_consensus.cc:2379] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate ad8a1cbdd8dc4e16b5779225fcccff22 in current term 1: Already voted for candidate 0fa161aa35264cbcb182beabddd7b55e in this term.
I20260811 12:13:11.802953 31982 raft_consensus.cc:3037] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:11.803246 32103 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [CANDIDATE]: Term 1 election: Requested vote from peers ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
I20260811 12:13:11.803876 31556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "0fa161aa35264cbcb182beabddd7b55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
I20260811 12:13:11.803754 31978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "0fa161aa35264cbcb182beabddd7b55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" is_pre_election: true
I20260811 12:13:11.804133 31556 raft_consensus.cc:2379] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0fa161aa35264cbcb182beabddd7b55e in current term 1: Already voted for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in this term.
I20260811 12:13:11.804600 31978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79949417ed2b4efc9e797438506fe2c2" candidate_uuid: "0fa161aa35264cbcb182beabddd7b55e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
I20260811 12:13:11.805160 31661 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0fa161aa35264cbcb182beabddd7b55e; no voters: ad8a1cbdd8dc4e16b5779225fcccff22, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:11.805426 32103 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 12:13:11.806066 31982 raft_consensus.cc:2454] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ad8a1cbdd8dc4e16b5779225fcccff22 in term 1.
I20260811 12:13:11.806382 31451 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22, b16569f2c44a4d04a2a46aecd0e1d066; no voters: 0fa161aa35264cbcb182beabddd7b55e
I20260811 12:13:11.806665 32104 raft_consensus.cc:2781] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:11.807055 32104 raft_consensus.cc:686] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 LEADER]: Becoming Leader. State: Replica: ad8a1cbdd8dc4e16b5779225fcccff22, State: Running, Role: LEADER
I20260811 12:13:11.807483 32104 consensus_queue.cc:237] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:11.811954 31338 catalog_manager.cc:5495] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 reported cstate change: term changed from 0 to 1, leader changed from <none> to ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } health_report { overall_health: UNKNOWN } } }
I20260811 12:13:11.832165 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.826925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36232 (local address 127.4.48.193:42591)
0811 12:13:11.827085 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.827091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.827179 (+ 88us) server_negotiation.cc:407] Connection header received
0811 12:13:11.827390 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.827394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.827467 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.827590 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:13:11.828004 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.828713 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.830076 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.830486 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.830592 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.831104 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:13:11.831571 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:13:11.831735 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.831819 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.831895 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:13:11.839721 31982 raft_consensus.cc:1261] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Refusing update from remote peer ad8a1cbdd8dc4e16b5779225fcccff22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:11.839883 31758 raft_consensus.cc:1261] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Refusing update from remote peer ad8a1cbdd8dc4e16b5779225fcccff22: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:11.840516 32116 consensus_queue.cc:1035] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:11.840852 32104 consensus_queue.cc:1035] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:11.852761 32120 mvcc.cc:205] Tried to move back new op lower bound from 7317300804966580224 to 7317300804846804992. Current Snapshot: MvccSnapshot[applied={T|T < 7317300804966580224}]
I20260811 12:13:11.869238 31339 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:11.869510 31339 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:11.872823 31339 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Sending DeleteTablet for 3 replicas of tablet 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:11.873889 31727 tablet_service.cc:1514] Processing DeleteTablet for tablet 79949417ed2b4efc9e797438506fe2c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:58666
I20260811 12:13:11.874096 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 31394
I20260811 12:13:11.874574 31949 tablet_service.cc:1514] Processing DeleteTablet for tablet 79949417ed2b4efc9e797438506fe2c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:37444
I20260811 12:13:11.874966 31527 tablet_service.cc:1514] Processing DeleteTablet for tablet 79949417ed2b4efc9e797438506fe2c2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:11 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:36216
I20260811 12:13:11.874837 32134 tablet_replica.cc:330] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: stopping tablet replica
I20260811 12:13:11.886902 32138 tablet_replica.cc:330] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: stopping tablet replica
I20260811 12:13:11.887385 32138 raft_consensus.cc:2229] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:11.888082 32138 raft_consensus.cc:2258] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:11.890549 32138 ts_tablet_manager.cc:1905] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:11.893963 32084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:11.875130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36244 (local address 127.4.48.193:42591)
0811 12:13:11.875293 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:11.875298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:11.875314 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:13:11.875364 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:11.875369 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:11.875491 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:11.875593 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:11.877437 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.878593 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.880645 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:11.881311 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:11.881453 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:11.885105 (+ 3652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:11.885120 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:11.885139 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:11.885198 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:11.889561 (+ 4363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.890465 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.890475 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.890479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.890636 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:11.891183 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:11.891188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:11.891192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:11.891715 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:11.892046 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:11.893297 (+ 1251us) server_negotiation.cc:299] Negotiation successful
0811 12:13:11.893532 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:13:11.894661 32134 raft_consensus.cc:2229] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:11.895531 31617 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:11.895824 32134 raft_consensus.cc:2258] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:11.898311 32134 ts_tablet_manager.cc:1905] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:11.898419 32138 ts_tablet_manager.cc:1918] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:11.898598 32138 log.cc:1196] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:11.899183 32138 ts_tablet_manager.cc:1939] T 79949417ed2b4efc9e797438506fe2c2 P b16569f2c44a4d04a2a46aecd0e1d066: Deleting consensus metadata
I20260811 12:13:11.901268 31316 catalog_manager.cc:4841] TS b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): tablet 79949417ed2b4efc9e797438506fe2c2 (table test-table [id=ab5ceca9df524a4e9dd460fdff3d0687]) successfully deleted
I20260811 12:13:11.906312 32134 ts_tablet_manager.cc:1918] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:11.906458 32134 log.cc:1196] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:11.906862 32134 ts_tablet_manager.cc:1939] T 79949417ed2b4efc9e797438506fe2c2 P 0fa161aa35264cbcb182beabddd7b55e: Deleting consensus metadata
I20260811 12:13:11.908394 31317 catalog_manager.cc:4841] TS 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): tablet 79949417ed2b4efc9e797438506fe2c2 (table test-table [id=ab5ceca9df524a4e9dd460fdff3d0687]) successfully deleted
I20260811 12:13:11.997066 32139 tablet_replica.cc:330] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: stopping tablet replica
I20260811 12:13:11.997359 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 31394
W20260811 12:13:12.024904 31316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.48.193:42591 (error 108)
I20260811 12:13:12.025187 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 31637
I20260811 12:13:12.026827 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.026415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36256)
0811 12:13:12.026588 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.026683 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 12:13:12.027077 31316 catalog_manager.cc:4584] TS ad8a1cbdd8dc4e16b5779225fcccff22 (127.4.48.193:42591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 79949417ed2b4efc9e797438506fe2c2: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
I20260811 12:13:12.039202 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.025985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58678 (local address 127.4.48.194:34189)
0811 12:13:12.026184 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:12.026190 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:12.026275 (+ 85us) server_negotiation.cc:407] Connection header received
0811 12:13:12.026477 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:12.026481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:12.026554 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:12.026650 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:12.028262 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:12.028999 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:12.030333 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:12.030731 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:12.030841 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:12.033347 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:12.033361 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:12.033376 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:12.033407 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:12.036391 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:12.037090 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:12.037100 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:12.037104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:12.037232 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:12.037600 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:12.037606 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:12.037610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:12.038028 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:12.038321 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:12.038653 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 12:13:12.038871 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:13:12.039865 31825 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:12.081041 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.080532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36262)
0811 12:13:12.080771 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.080870 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 12:13:12.082103 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.081747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36278)
0811 12:13:12.081901 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.081975 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:13:12.129774 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 31637
I20260811 12:13:12.142690 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.142249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36290)
0811 12:13:12.142450 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.142537 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 12:13:12.143713 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.143342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36298)
0811 12:13:12.143495 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.143569 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:13:12.158553 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 31843
I20260811 12:13:12.171046 32086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.159233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37458 (local address 127.4.48.195:39259)
0811 12:13:12.159417 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:12.159421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:12.159515 (+ 94us) server_negotiation.cc:407] Connection header received
0811 12:13:12.159660 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:12.159664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:12.159790 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:12.159884 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:12.161145 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:12.161832 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:12.163148 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:12.163582 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:12.163740 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:12.166136 (+ 2396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:12.166152 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:12.166170 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:12.166207 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:12.168690 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:12.169340 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:12.169348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:12.169350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:12.169490 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:12.169802 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:12.169805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:12.169806 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:12.170161 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:12.170376 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:12.170604 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 12:13:12.170753 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:13:12.171657 32053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:12.223971 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.223515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36314)
0811 12:13:12.223731 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.223818 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:13:12.225044 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.224640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36330)
0811 12:13:12.224854 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.224928 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:13:12.271941 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 31843
I20260811 12:13:12.298938 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 31289
I20260811 12:13:12.300120 31384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:12.411940 32087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:12.411220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:42591 (local address 127.0.0.1:36342)
0811 12:13:12.411576 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:12.411718 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189}
I20260811 12:13:12.411984 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 31289
I20260811 12:13:32.442488 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--webserver_interface=127.4.48.254
--webserver_port=41837
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:32.775341 32151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:32.775614 32151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:32.775823 32151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:32.783025 32151 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 12:13:32.783126 32151 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 12:13:32.783195 32151 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 12:13:32.783268 32151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:13:32.783344 32151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:32.783435 32151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:32.783494 32151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:13:32.783568 32151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:13:32.793141 32151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=41837
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:32.794345 32151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:32.795636 32151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:32.807200 32157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:32.807163 32158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:32.807233 32161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:32.807979 32151 server_base.cc:1034] running on GCE node
I20260811 12:13:32.808681 32151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:32.809955 32151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:32.811177 32151 hybrid_clock.cc:648] HybridClock initialized: now 1786450412811111 us; error 31 us; skew 500 ppm
I20260811 12:13:32.811535 32151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:32 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450412, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:32.818540 32151 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:13:32.820036 32151 webserver.cc:460] Webserver started at http://127.4.48.254:41837/ using document root <none> and password file <none>
I20260811 12:13:32.820719 32151 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:32.820807 32151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:32.826350 32151 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:13:32.829164 32170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:32.830490 32151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:13:32.830637 32151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "ff34618a7b44452eaafc68f7b14587e9"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "166146d51d6c14275db74766bc20c4ee"
server_key_iv: "4106a5b0c2ad758347b8bdc5292c7b64"
server_key_version: "encryptionkey@0"
I20260811 12:13:32.831589 32151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:32.852977 32151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:32.856959 32151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:32.857234 32151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:32.874869 32151 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39031
I20260811 12:13:32.874905 32264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39031 every 8 connection(s)
I20260811 12:13:32.876279 32151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:13:32.880463 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 32151
I20260811 12:13:32.881283 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:42591
--local_ip_for_outbound_sockets=127.4.48.193
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=41777
--webserver_interface=127.4.48.193
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:13:32.883102 32267 sys_catalog.cc:263] Verifying existing consensus state
I20260811 12:13:32.886323 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap starting.
I20260811 12:13:32.909564 32267 log.cc:824] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:32.925756 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:13:32.926268 32267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Bootstrap complete.
I20260811 12:13:32.935271 32267 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:32.936293 32267 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Initialized, Role: FOLLOWER
I20260811 12:13:32.936944 32267 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:32.937162 32267 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:13:32.937276 32267 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:13:32.937444 32267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 2 FOLLOWER]: Advancing to term 3
I20260811 12:13:32.941200 32267 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:32.941658 32267 leader_election.cc:302] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff34618a7b44452eaafc68f7b14587e9; no voters:
I20260811 12:13:32.942281 32267 leader_election.cc:288] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 12:13:32.942425 32271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 12:13:32.943569 32271 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [term 3 LEADER]: Becoming Leader. State: Replica: ff34618a7b44452eaafc68f7b14587e9, State: Running, Role: LEADER
I20260811 12:13:32.944190 32271 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } }
I20260811 12:13:32.944404 32267 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:13:32.946949 32274 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff34618a7b44452eaafc68f7b14587e9. Latest consensus state: current_term: 3 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:13:32.947039 32272 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ff34618a7b44452eaafc68f7b14587e9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff34618a7b44452eaafc68f7b14587e9" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39031 } } }
I20260811 12:13:32.947328 32274 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:32.947404 32272 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:32.959394 32280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:13:32.967957 32280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:13:32.969384 32280 catalog_manager.cc:1259] Loaded cluster ID: 27be41276b2d47b69c716df0f52cba73
I20260811 12:13:32.969497 32280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:13:32.973467 32280 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:13:32.975275 32280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Loaded TSK: 0
I20260811 12:13:32.975417 32280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff34618a7b44452eaafc68f7b14587e9: Loaded TSK: 1
F20260811 12:13:32.976506 32280 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0a189319ad google::LogMessage::Fail() at ??:0
@ 0x7f0a1893488a google::LogMessage::SendToLog() at ??:0
@ 0x7f0a189314de google::LogMessage::Flush() at ??:0
@ 0x7f0a18932a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0a1c6957d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0a1c68473e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0a2e3641c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0a2e363783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0a2e3ef2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f0a2e3ef0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f0a2e4395e9 std::function<>::operator()() at ??:0
@ 0x7f0a2e367cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f0a2e365c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0a2e3ed610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f0a2e3ed439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0a1b57d345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0a1b58091c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f0a1b580799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f0a1b55beaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0a1e8356db start_thread at ??:0
@ 0x7f0a177cb61f clone at ??:0
W20260811 12:13:33.305644 32269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:33.305927 32269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:33.306051 32269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:33.313139 32269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:33.313238 32269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:33.313402 32269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:13:33.322804 32269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:42591
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=41777
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:33.324126 32269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:33.325515 32269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:33.338223 32339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:33.338246 32342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:33.338328 32340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:33.339373 32269 server_base.cc:1034] running on GCE node
I20260811 12:13:33.339983 32269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:33.340859 32269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:33.342046 32269 hybrid_clock.cc:648] HybridClock initialized: now 1786450413342000 us; error 33 us; skew 500 ppm
I20260811 12:13:33.342348 32269 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:33 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450413, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:33.345419 32269 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:13:33.346745 32269 webserver.cc:460] Webserver started at http://127.4.48.193:41777/ using document root <none> and password file <none>
I20260811 12:13:33.347370 32269 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:33.347470 32269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:33.352761 32269 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:13:33.355580 32352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:33.356894 32269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:33.357043 32269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "ad8a1cbdd8dc4e16b5779225fcccff22"
format_stamp: "Formatted at 2026-08-11 12:12:47 on dist-test-slave-nj21"
server_key: "1ab21ab311290eda4aebcae9cde5d3e9"
server_key_iv: "b25e6158667173c26a3cf9866ffab25d"
server_key_version: "encryptionkey@0"
I20260811 12:13:33.358018 32269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:33.374131 32269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:33.377768 32269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:33.378005 32269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:33.378921 32269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:33.380641 32364 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 12:13:33.386111 32269 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 12:13:33.386212 32269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260811 12:13:33.386339 32269 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 12:13:33.389457 32269 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 12:13:33.389531 32269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260811 12:13:33.389853 32364 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Bootstrap starting.
I20260811 12:13:33.416069 32364 log.cc:824] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:33.427414 32364 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 12:13:33.428064 32364 tablet_bootstrap.cc:492] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Bootstrap complete.
I20260811 12:13:33.429428 32364 ts_tablet_manager.cc:1397] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent bootstrapping tablet: real 0.040s user 0.016s sys 0.021s
I20260811 12:13:33.430751 32269 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:42591
I20260811 12:13:33.430819 32541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:42591 every 8 connection(s)
I20260811 12:13:33.432041 32269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:13:33.436980 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.434884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:58075)
0811 12:13:33.435841 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.436074 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":636,"thread_start_us":408,"threads_started":1}
W20260811 12:13:33.437541 32545 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:39031 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:39031: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
I20260811 12:13:33.438408 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.438100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:50131)
0811 12:13:33.438234 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.438306 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:13:33.437656 32364 raft_consensus.cc:348] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:33.438752 32364 raft_consensus.cc:729] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad8a1cbdd8dc4e16b5779225fcccff22, State: Initialized, Role: FOLLOWER
I20260811 12:13:33.439316 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.438938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:54197)
0811 12:13:33.439068 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.439180 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 12:13:33.439363 32364 consensus_queue.cc:260] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
W20260811 12:13:33.439553 32545 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:13:33.440227 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 32269
I20260811 12:13:33.441007 32364 ts_tablet_manager.cc:1428] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 12:13:33.441067 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:34189
--local_ip_for_outbound_sockets=127.4.48.194
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=40845
--webserver_interface=127.4.48.194
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:13:33.441813 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.441522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:37029)
0811 12:13:33.441646 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.441709 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 12:13:33.442584 32548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.442304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:35207)
0811 12:13:33.442421 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.442480 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:13:33.771378 32554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:33.771647 32554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:33.771817 32554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:33.778791 32554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:33.778898 32554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:33.779048 32554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:13:33.788293 32554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:34189
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=40845
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:33.789520 32554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:33.790803 32554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:33.803252 32563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:33.803283 32569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:33.803205 32566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:33.804395 32554 server_base.cc:1034] running on GCE node
I20260811 12:13:33.804999 32554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:33.805881 32554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:33.807081 32554 hybrid_clock.cc:648] HybridClock initialized: now 1786450413807030 us; error 42 us; skew 500 ppm
I20260811 12:13:33.807370 32554 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:33 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450413, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:33.810389 32554 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:13:33.811710 32554 webserver.cc:460] Webserver started at http://127.4.48.194:40845/ using document root <none> and password file <none>
I20260811 12:13:33.812351 32554 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:33.812469 32554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:33.817462 32554 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 12:13:33.819977 32577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:33.821239 32554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:13:33.821401 32554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "0fa161aa35264cbcb182beabddd7b55e"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "2222de6eaea72af179fb37fed5ea8ef9"
server_key_iv: "f945078d46d520e576f810c60ce47fb3"
server_key_version: "encryptionkey@0"
I20260811 12:13:33.822347 32554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:33.846824 32554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:33.850416 32554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:33.850659 32554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:33.851646 32554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:33.853041 32554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:33.853127 32554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:33.853204 32554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:33.853266 32554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:33.882757 32554 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:34189
I20260811 12:13:33.882830 32763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:34189 every 8 connection(s)
I20260811 12:13:33.884178 32554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:13:33.887712 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 32554
I20260811 12:13:33.888444 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:39259
--local_ip_for_outbound_sockets=127.4.48.195
--tserver_master_addrs=127.4.48.254:39031
--webserver_port=42477
--webserver_interface=127.4.48.195
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:41985
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:13:33.889034 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.887041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:44483)
0811 12:13:33.887894 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.888132 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":334,"threads_started":1}
W20260811 12:13:33.889660 32764 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:39031 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:39031: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
I20260811 12:13:33.890465 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.890164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:52331)
0811 12:13:33.890297 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.890361 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:13:33.891208 32766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:33.890926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:44987)
0811 12:13:33.891036 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:33.891106 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:13:33.891417 32764 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 12:13:34.226078 32767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:34.226364 32767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:34.226472 32767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:34.233421 32767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:34.233520 32767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:34.233635 32767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:13:34.243014 32767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:41985
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:39259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=42477
--tserver_master_addrs=127.4.48.254:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:34.244259 32767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:34.245525 32767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:34.257958 314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:34.258049 308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:34.258005 311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:34.258792 32767 server_base.cc:1034] running on GCE node
I20260811 12:13:34.259364 32767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:34.260269 32767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:34.261469 32767 hybrid_clock.cc:648] HybridClock initialized: now 1786450414261420 us; error 47 us; skew 500 ppm
I20260811 12:13:34.261767 32767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:34 dist-test-slave-nj21 krb5kdc[30355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450414, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:34.264676 32767 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:13:34.266088 32767 webserver.cc:460] Webserver started at http://127.4.48.195:42477/ using document root <none> and password file <none>
I20260811 12:13:34.266712 32767 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:34.266839 32767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:34.271905 32767 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:34.274443 327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:34.275646 32767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:13:34.275835 32767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "b16569f2c44a4d04a2a46aecd0e1d066"
format_stamp: "Formatted at 2026-08-11 12:12:48 on dist-test-slave-nj21"
server_key: "1a36f690ddb0592f14735fc09107ce8c"
server_key_iv: "6e386f9163f0f5d517da715eacf3f7c6"
server_key_version: "encryptionkey@0"
I20260811 12:13:34.276737 32767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:34.311473 32767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:34.315092 32767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:34.315335 32767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:34.316299 32767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:34.317570 32767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:34.317664 32767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:34.317782 32767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:34.317853 32767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:34.347957 32767 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39259
I20260811 12:13:34.348039 532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39259 every 8 connection(s)
I20260811 12:13:34.349313 32767 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:13:34.353859 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.352096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:55673)
0811 12:13:34.352841 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.353056 (+ 215us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":271,"threads_started":1}
W20260811 12:13:34.354504 533 heartbeater.cc:644] Failed to heartbeat to 127.4.48.254:39031 (0 consecutive failures): Network error: Failed to ping master at 127.4.48.254:39031: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
I20260811 12:13:34.355249 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.354951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:34537)
0811 12:13:34.355091 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.355155 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 12:13:34.356062 537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.355725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:52223)
0811 12:13:34.355833 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.355922 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260811 12:13:34.356295 533 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 12:13:34.356303 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 32767
I20260811 12:13:34.444088 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.443298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:60077)
0811 12:13:34.443792 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.443875 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":222,"threads_started":1}
I20260811 12:13:34.768378 540 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:34.768674 540 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:34.770632 540 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
Aug 11 12:13:34 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450413, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
Aug 11 12:13:34 dist-test-slave-nj21 krb5kdc[30355](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450413, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:13:34.791148 539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.771024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:34189 (local address 127.4.48.193:55843)
0811 12:13:34.771344 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.771404 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:34.772014 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:34.775516 (+ 3502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:34.775527 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:34.776381 (+ 854us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:34.777091 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:34.777109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.778655 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.778659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:34.780395 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:34.780410 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.781443 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.781451 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:34.781660 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:34.782396 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:34.782413 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:34.784730 (+ 2317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:34.787923 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:34.787931 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:34.787945 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:34.788421 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:34.788834 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:34.788840 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:34.788844 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:34.789012 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:34.789749 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:34.789756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:34.790039 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 12:13:34.790269 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 12:13:34.790752 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":168,"spinlock_wait_cycles":34432}
I20260811 12:13:34.791294 543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.771185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:55843 (local address 127.4.48.194:34189)
0811 12:13:34.772794 (+ 1609us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:34.772811 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:34.772852 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:13:34.774624 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:34.774633 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:34.775018 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:34.775348 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:34.777276 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.778474 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.780538 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.781229 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.781415 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:34.785007 (+ 3592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:34.785015 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:34.785027 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:34.785066 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:34.787797 (+ 2731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:34.788555 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:34.788564 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:34.788570 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:34.788723 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:34.789129 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:34.789134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:34.789137 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:34.789614 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:34.789803 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:34.790529 (+ 726us) server_negotiation.cc:299] Negotiation successful
0811 12:13:34.790914 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1413,"thread_start_us":1261,"threads_started":1}
I20260811 12:13:34.791311 548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.771405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:40991 (local address 127.4.48.195:39259)
0811 12:13:34.773171 (+ 1766us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:34.773185 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:34.773523 (+ 338us) server_negotiation.cc:407] Connection header received
0811 12:13:34.775064 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:34.775073 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:34.775386 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:34.775712 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:34.777280 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.778521 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.780534 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.781199 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.781372 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:34.785278 (+ 3906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:34.785285 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:34.785298 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:34.785343 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:34.787970 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:34.788686 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:34.788692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:34.788698 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:34.788843 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:34.789235 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:34.789239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:34.789241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:34.789703 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:34.789874 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:34.790536 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 12:13:34.790930 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1573,"thread_start_us":1410,"threads_started":1}
I20260811 12:13:34.792052 541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.771093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39259 (local address 127.4.48.193:40991)
0811 12:13:34.773411 (+ 2318us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.773429 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:34.773554 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:34.775688 (+ 2134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:34.775694 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:34.776381 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:34.777091 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:34.777109 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.778671 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.778675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:34.780401 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:34.780416 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:34.781400 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:34.781410 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:34.781658 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:34.782337 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:34.782359 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:34.785023 (+ 2664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:34.788094 (+ 3071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:34.788099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:34.788103 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:34.788555 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:34.788962 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:34.788967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:34.788971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:34.789126 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:34.789829 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:34.789835 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:34.790037 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 12:13:34.790271 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 12:13:34.791700 (+ 1429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2186,"mutex_wait_us":1187,"spinlock_wait_cycles":7424,"thread_start_us":2037,"threads_started":1}
W20260811 12:13:34.794593 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:34.794839 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:34.794978 32355 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:34.795322 540 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:34.892726 550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:34.891928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:39623)
0811 12:13:34.892342 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:34.892495 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":181,"threads_started":1}
I20260811 12:13:35.357585 551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:35.356737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:51545)
0811 12:13:35.357190 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:35.357319 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":223,"threads_started":1}
I20260811 12:13:35.445523 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:35.444894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:44213)
0811 12:13:35.445290 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:35.445367 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":204,"threads_started":1}
I20260811 12:13:35.894207 556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:35.893560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:39299)
0811 12:13:35.893981 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:35.894056 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":233,"threads_started":1}
I20260811 12:13:36.359088 559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:36.358402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:49489)
0811 12:13:36.358819 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:36.358919 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":208,"threads_started":1}
I20260811 12:13:36.447098 560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:36.446284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:46677)
0811 12:13:36.446822 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:36.446908 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436,"thread_start_us":253,"threads_started":1}
I20260811 12:13:36.555227 561 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:36.555430 561 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:36.556110 561 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
W20260811 12:13:36.557051 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:36.557231 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:36.557333 32355 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:36.557564 561 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:36.895645 562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:36.894972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:48887)
0811 12:13:36.895406 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:36.895485 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260811 12:13:37.360620 564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:37.359926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:52229)
0811 12:13:37.360356 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:37.360459 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
I20260811 12:13:37.448623 566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:37.447942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:37621)
0811 12:13:37.448363 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:37.448447 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":198,"threads_started":1}
I20260811 12:13:37.897125 574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:37.896425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:46809)
0811 12:13:37.896857 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:37.896945 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":212,"threads_started":1}
I20260811 12:13:38.077090 576 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:38.077273 576 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:38.077841 576 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
W20260811 12:13:38.078913 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:38.079110 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:38.079224 32355 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:38.079491 576 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:38.362152 577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:38.361416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:44653)
0811 12:13:38.361893 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:38.361973 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":229,"threads_started":1}
I20260811 12:13:38.450529 578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:38.449704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:40197)
0811 12:13:38.450231 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:38.450328 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":264,"threads_started":1}
I20260811 12:13:38.898671 579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:38.897974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:52021)
0811 12:13:38.898423 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:38.898501 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260811 12:13:39.363754 582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:39.362979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:51311)
0811 12:13:39.363473 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:39.363557 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":262,"threads_started":1}
I20260811 12:13:39.452207 583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:39.451397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:44491)
0811 12:13:39.451928 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:39.452049 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":279,"threads_started":1}
I20260811 12:13:39.900696 584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:39.899524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:50651)
0811 12:13:39.900374 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:39.900473 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":732,"thread_start_us":614,"threads_started":1}
I20260811 12:13:40.139793 585 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:40.139995 585 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:40.140645 585 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
W20260811 12:13:40.141666 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:40.141886 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:40.141987 32355 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:40.142267 585 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:40.365239 586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:40.364522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:33447)
0811 12:13:40.364970 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:40.365045 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":242,"threads_started":1}
I20260811 12:13:40.454005 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:40.453132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:48575)
0811 12:13:40.453657 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:40.453771 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":261,"threads_started":1}
I20260811 12:13:40.902449 588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:40.901658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:37849)
0811 12:13:40.902169 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:40.902269 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":220,"threads_started":1}
I20260811 12:13:41.367048 589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:41.366204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:57633)
0811 12:13:41.366764 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:41.366850 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":227,"threads_started":1}
I20260811 12:13:41.455547 592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:41.454866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:58073)
0811 12:13:41.455311 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:41.455392 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":223,"threads_started":1}
I20260811 12:13:41.903934 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:41.903251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:60281)
0811 12:13:41.903637 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:41.903742 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":196,"threads_started":1}
I20260811 12:13:42.368549 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:42.367866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:52713)
0811 12:13:42.368296 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:42.368374 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":215,"threads_started":1}
I20260811 12:13:42.457141 599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:42.456394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:43955)
0811 12:13:42.456870 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:42.456959 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":260,"threads_started":1}
I20260811 12:13:42.905503 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:42.904756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:39509)
0811 12:13:42.905236 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:42.905331 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":213,"threads_started":1}
I20260811 12:13:43.370204 601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:43.369372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:52369)
0811 12:13:43.369953 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:43.370044 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":273,"threads_started":1}
I20260811 12:13:43.458719 604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:43.458035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:52667)
0811 12:13:43.458477 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:43.458555 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":235,"threads_started":1}
I20260811 12:13:43.704663 606 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:43.704916 606 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:43.705619 606 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
W20260811 12:13:43.706477 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:43.706658 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:43.706784 32353 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:43.707037 606 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:43.907114 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:43.906399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:59533)
0811 12:13:43.906852 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:43.906932 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":225,"threads_started":1}
I20260811 12:13:44.371758 610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:44.371063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:54333)
0811 12:13:44.371478 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:44.371554 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260811 12:13:44.460335 611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:44.459611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:39299)
0811 12:13:44.460075 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:44.460154 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":269,"threads_started":1}
I20260811 12:13:44.911571 612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:44.909860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:58971)
0811 12:13:44.911269 (+ 1409us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:44.911358 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1279,"thread_start_us":1140,"threads_started":1}
I20260811 12:13:45.373226 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:45.372545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:55797)
0811 12:13:45.372987 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:45.373072 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":230,"threads_started":1}
I20260811 12:13:45.461851 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:45.461159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:48587)
0811 12:13:45.461580 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:45.461659 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":212,"threads_started":1}
I20260811 12:13:45.913213 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:45.912551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:35033)
0811 12:13:45.912960 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:45.913038 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":212,"threads_started":1}
I20260811 12:13:46.374716 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:46.374029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:55217)
0811 12:13:46.374471 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:46.374547 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":237,"threads_started":1}
I20260811 12:13:46.463352 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:46.462646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:56873)
0811 12:13:46.463082 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:46.463170 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":227,"threads_started":1}
I20260811 12:13:46.914734 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:46.914042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:53917)
0811 12:13:46.914481 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:46.914569 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":219,"threads_started":1}
I20260811 12:13:47.376288 622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:47.375556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:41967)
0811 12:13:47.376027 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:47.376116 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":245,"threads_started":1}
I20260811 12:13:47.464943 623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:47.464137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:39725)
0811 12:13:47.464598 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:47.464705 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":231,"threads_started":1}
I20260811 12:13:47.876438 625 raft_consensus.cc:482] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:47.876662 625 raft_consensus.cc:504] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ad8a1cbdd8dc4e16b5779225fcccff22" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 42591 } } peers { permanent_uuid: "0fa161aa35264cbcb182beabddd7b55e" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 34189 } } peers { permanent_uuid: "b16569f2c44a4d04a2a46aecd0e1d066" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39259 } }
I20260811 12:13:47.877391 625 leader_election.cc:288] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189), b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259)
W20260811 12:13:47.878355 32353 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b16569f2c44a4d04a2a46aecd0e1d066 (127.4.48.195:39259): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
W20260811 12:13:47.878563 32355 leader_election.cc:341] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0fa161aa35264cbcb182beabddd7b55e (127.4.48.194:34189): Not found: Tablet not found: 79949417ed2b4efc9e797438506fe2c2
I20260811 12:13:47.878722 32355 leader_election.cc:302] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ad8a1cbdd8dc4e16b5779225fcccff22; no voters: 0fa161aa35264cbcb182beabddd7b55e, b16569f2c44a4d04a2a46aecd0e1d066
I20260811 12:13:47.878986 625 raft_consensus.cc:2726] T 79949417ed2b4efc9e797438506fe2c2 P ad8a1cbdd8dc4e16b5779225fcccff22 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 12:13:47.916383 626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:47.915586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:43421)
0811 12:13:47.916077 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:47.916156 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":270,"threads_started":1}
I20260811 12:13:48.377852 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:48.377112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:53783)
0811 12:13:48.377574 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:48.377655 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":242,"threads_started":1}
I20260811 12:13:48.466754 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:48.466074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.193:34015)
0811 12:13:48.466492 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:48.466580 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":201,"threads_started":1}
I20260811 12:13:48.918145 630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:48.917441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.194:45135)
0811 12:13:48.917869 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:48.917981 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":218,"threads_started":1}
I20260811 12:13:49.358340 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 32269
I20260811 12:13:49.376178 632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:49.360106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50104 (local address 127.4.48.193:42591)
0811 12:13:49.360590 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:49.360601 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:49.360632 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:13:49.360695 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:49.360701 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:49.360931 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:49.361060 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:49.362534 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.363518 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.364877 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.365448 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.365552 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:49.371239 (+ 5687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:49.371251 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:49.371264 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:49.371301 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:49.373811 (+ 2510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.374420 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.374426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.374429 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.374560 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.374836 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.374839 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.374841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.375194 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:49.375406 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:49.375661 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:13:49.375916 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":517,"threads_started":1}
I20260811 12:13:49.377343 32535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:49.394161 634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:49.378903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:39031 (local address 127.4.48.195:37367)
0811 12:13:49.393893 (+ 14990us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:49.393985 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.254:39031: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14874,"threads_started":1}
I20260811 12:13:49.464711 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 32269
I20260811 12:13:49.489643 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 32554
I20260811 12:13:49.502526 635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:49.490397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43872 (local address 127.4.48.194:34189)
0811 12:13:49.490893 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:49.490897 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:49.490913 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:13:49.490969 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:49.490973 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:49.491035 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:49.491133 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:49.492513 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.493276 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.494467 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.494908 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.495021 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:49.497604 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:49.497609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:49.497613 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:49.497647 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:49.499990 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.500625 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.500630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.500632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.500770 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.501115 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.501119 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.501121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.501390 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:49.501642 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:49.501948 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 12:13:49.502167 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":225,"threads_started":1}
I20260811 12:13:49.503252 32758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:49.584208 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 32554
I20260811 12:13:49.608615 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 32767
I20260811 12:13:49.621896 637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:49.609391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36762 (local address 127.4.48.195:39259)
0811 12:13:49.609958 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:49.609962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:49.609980 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:13:49.610035 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:49.610038 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:49.610100 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:49.610195 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:49.611784 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.612612 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.613848 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:49.614285 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:49.614397 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:49.616921 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:49.616926 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:49.616929 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:49.616963 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:49.619322 (+ 2359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.619988 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.619994 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.619998 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.620147 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:49.620516 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:49.620520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:49.620522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:49.620786 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:49.621044 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:49.621297 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 12:13:49.621523 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":270,"threads_started":1}
I20260811 12:13:49.622604 527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:49.705288 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 32767
2026-08-11T12:13:49Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (64391 ms)
[----------] 30 tests from SecurityITest (296802 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:13:49 dist-test-slave-nj21 krb5kdc[643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:13:49 dist-test-slave-nj21 krb5kdc[643](info): set up 2 sockets
Aug 11 12:13:49 dist-test-slave-nj21 krb5kdc[643](info): commencing operation
krb5kdc: starting...
W20260811 12:13:50.944339 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.194s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:13:51 dist-test-slave-nj21 krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:13:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:13:51.214835 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:41639
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:51.543793 662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:51.544059 662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:51.544191 662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:51.551144 662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:13:51.551239 662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:51.551316 662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:51.551398 662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:13:51.551447 662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:13:51.560679 662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:41639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:51.561913 662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:51.563169 662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:51.574401 670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:51.574348 672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:51.574869 668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:51.575382 662 server_base.cc:1034] running on GCE node
I20260811 12:13:51.576190 662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:51.577500 662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:51.578698 662 hybrid_clock.cc:648] HybridClock initialized: now 1786450431578650 us; error 32 us; skew 500 ppm
I20260811 12:13:51.579005 662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:51 dist-test-slave-nj21 krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:51.581920 662 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:13:51.583247 662 webserver.cc:460] Webserver started at http://127.4.48.254:34965/ using document root <none> and password file <none>
I20260811 12:13:51.583907 662 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:51.584010 662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:51.584280 662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:51.587090 662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "f9bd02f63051411e83c3272815898878"
format_stamp: "Formatted at 2026-08-11 12:13:51 on dist-test-slave-nj21"
server_key: "528ab0556b43f20e3b731f5b50a22509"
server_key_iv: "22ed831a51275a37a325ba5c807f4c7d"
server_key_version: "encryptionkey@0"
I20260811 12:13:51.587841 662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "f9bd02f63051411e83c3272815898878"
format_stamp: "Formatted at 2026-08-11 12:13:51 on dist-test-slave-nj21"
server_key: "528ab0556b43f20e3b731f5b50a22509"
server_key_iv: "22ed831a51275a37a325ba5c807f4c7d"
server_key_version: "encryptionkey@0"
I20260811 12:13:51.592921 662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260811 12:13:51.596066 684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:51.597256 662 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.002s
I20260811 12:13:51.597417 662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "f9bd02f63051411e83c3272815898878"
format_stamp: "Formatted at 2026-08-11 12:13:51 on dist-test-slave-nj21"
server_key: "528ab0556b43f20e3b731f5b50a22509"
server_key_iv: "22ed831a51275a37a325ba5c807f4c7d"
server_key_version: "encryptionkey@0"
I20260811 12:13:51.597579 662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:51.613358 662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:51.616894 662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:51.617132 662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:51.633792 662 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:41639
I20260811 12:13:51.633826 770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:41639 every 8 connection(s)
I20260811 12:13:51.635165 662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:13:51.638782 771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:51.640477 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 662
I20260811 12:13:51.640679 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:13:51.641465 4291 external_mini_cluster.cc:1428] Setting key 78a09a7f4169d824115935717a880f23
I20260811 12:13:51.652503 771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: Bootstrap starting.
Aug 11 12:13:51 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:51.656180 771 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:51.657370 771 log.cc:824] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:51.660365 771 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: No bootstrap required, opened a new log
I20260811 12:13:51.664870 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:51.643429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.4.48.254:41639)
0811 12:13:51.644170 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:51.644183 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:51.644234 (+ 51us) server_negotiation.cc:407] Connection header received
0811 12:13:51.645758 (+ 1524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:51.645779 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:51.646125 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:51.646417 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:51.648198 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:51.649567 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:51.651488 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:51.652189 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:51.652338 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:51.657143 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:51.657152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:51.657166 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:51.657227 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:51.660577 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:51.661434 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:51.661440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:51.661447 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:51.661633 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:51.662079 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:51.662084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:51.662086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:51.662673 (+ 587us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:51.662929 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:51.663405 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 12:13:51.663798 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":349,"threads_started":1}
I20260811 12:13:51.668953 771 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } }
I20260811 12:13:51.669269 771 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:51.669370 771 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9bd02f63051411e83c3272815898878, State: Initialized, Role: FOLLOWER
I20260811 12:13:51.669870 771 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } }
I20260811 12:13:51.670085 771 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:13:51.670209 771 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:13:51.670362 771 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:51.672428 771 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } }
I20260811 12:13:51.672870 771 leader_election.cc:302] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9bd02f63051411e83c3272815898878; no voters:
I20260811 12:13:51.673455 771 leader_election.cc:288] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:13:51.673560 781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:51.674554 781 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [term 1 LEADER]: Becoming Leader. State: Replica: f9bd02f63051411e83c3272815898878, State: Running, Role: LEADER
I20260811 12:13:51.675155 781 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } }
I20260811 12:13:51.675642 771 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:13:51.679829 782 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9bd02f63051411e83c3272815898878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } } }
I20260811 12:13:51.680202 782 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:51.681452 783 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9bd02f63051411e83c3272815898878. Latest consensus state: current_term: 1 leader_uuid: "f9bd02f63051411e83c3272815898878" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9bd02f63051411e83c3272815898878" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 41639 } } }
I20260811 12:13:51.681766 783 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878 [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:51.687981 789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:13:51.692660 789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:13:51.700521 789 catalog_manager.cc:1347] Generated new cluster ID: f71a53e9c2cb4fa88e272ac527b747e9
I20260811 12:13:51.700628 789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:13:51.735283 789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:13:51.737247 789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:13:51.746924 789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: Generated new TSK 0
I20260811 12:13:51.747943 789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:13:51.815833 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:52.143808 809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:52.144085 809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:52.144212 809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:52.151216 809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:52.151335 809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:52.151489 809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:13:52.160815 809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:52.162065 809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:52.163333 809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:52.175729 819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:52.176165 823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:52.176221 818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:52.176532 809 server_base.cc:1034] running on GCE node
I20260811 12:13:52.177145 809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:52.177994 809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:52.179184 809 hybrid_clock.cc:648] HybridClock initialized: now 1786450432179138 us; error 42 us; skew 500 ppm
I20260811 12:13:52.179473 809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:52 dist-test-slave-nj21 krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450432, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:52.182451 809 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:13:52.183884 809 webserver.cc:460] Webserver started at http://127.4.48.193:43361/ using document root <none> and password file <none>
I20260811 12:13:52.184507 809 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:52.184623 809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:52.184921 809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:52.187654 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "235bcce3d1b34292a0dad338b9138c3c"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "d38fa78ef72d91d34527780b309e3ccd"
server_key_iv: "d1f896c71124b6dbe1a9bb242b802252"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.188573 809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "235bcce3d1b34292a0dad338b9138c3c"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "d38fa78ef72d91d34527780b309e3ccd"
server_key_iv: "d1f896c71124b6dbe1a9bb242b802252"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.193602 809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 12:13:52.196802 843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.198029 809 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:52.198174 809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "235bcce3d1b34292a0dad338b9138c3c"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "d38fa78ef72d91d34527780b309e3ccd"
server_key_iv: "d1f896c71124b6dbe1a9bb242b802252"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.198351 809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:52.247153 809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:52.251188 809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:52.251456 809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:52.252422 809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:52.253721 809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:52.253809 809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.253935 809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:52.253990 809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.283836 809 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:34601
I20260811 12:13:52.283878 1031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:34601 every 8 connection(s)
I20260811 12:13:52.285039 809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:13:52.292977 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 809
I20260811 12:13:52.293226 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:13:52.293965 4291 external_mini_cluster.cc:1428] Setting key f9a58da4dd07bbf96f0d52211ab416e7
Aug 11 12:13:52 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450432, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:52.305194 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:52.287396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:48149 (local address 127.4.48.254:41639)
0811 12:13:52.287897 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:52.287903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:52.288782 (+ 879us) server_negotiation.cc:407] Connection header received
0811 12:13:52.290656 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:52.290661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:52.290727 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:52.290811 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:52.292303 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.293384 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.295152 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.295581 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.295724 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:52.299503 (+ 3779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:52.299508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:52.299511 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:52.299545 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:52.302122 (+ 2577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:52.303008 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:52.303011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:52.303014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:52.303127 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:52.303558 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:52.303560 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:52.303562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:52.303876 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:52.303985 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:52.304780 (+ 795us) server_negotiation.cc:299] Negotiation successful
0811 12:13:52.304950 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":210,"threads_started":1}
I20260811 12:13:52.306497 1038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:52.287923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41639 (local address 127.4.48.193:48149)
0811 12:13:52.288594 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:52.288661 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:52.290409 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:52.290998 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:52.291006 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:52.291496 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:52.292108 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:52.292126 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.293563 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.293568 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:52.295000 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:52.295013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.295850 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.295859 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:52.296131 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:52.296794 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:52.296817 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:52.299284 (+ 2467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:52.302279 (+ 2995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:52.302287 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:52.302306 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:52.302873 (+ 567us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:52.303222 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:52.303226 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:52.303229 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:52.303467 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:52.303997 (+ 530us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:52.304005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:52.304405 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 12:13:52.304687 (+ 282us) client_negotiation.cc:240] Negotiation successful
0811 12:13:52.305197 (+ 510us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":266,"threads_started":1}
I20260811 12:13:52.307943 1032 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41639
I20260811 12:13:52.308261 1032 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:52.309103 1032 heartbeater.cc:511] Master 127.4.48.254:41639 requested a full tablet report, sending...
I20260811 12:13:52.311427 716 ts_manager.cc:194] Registered new tserver with Master: 235bcce3d1b34292a0dad338b9138c3c (127.4.48.193:34601)
I20260811 12:13:52.313334 716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:48149
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:13:52.356345 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:52.684015 1045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:52.684290 1045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:52.684428 1045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:52.691440 1045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:52.691550 1045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:52.691715 1045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:13:52.701253 1045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:52.702615 1045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:52.703943 1045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:52.716228 1060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:52.716279 1055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:52.716313 1056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:52.717170 1045 server_base.cc:1034] running on GCE node
I20260811 12:13:52.717705 1045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:52.718539 1045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:52.719769 1045 hybrid_clock.cc:648] HybridClock initialized: now 1786450432719709 us; error 44 us; skew 500 ppm
I20260811 12:13:52.720063 1045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:52 dist-test-slave-nj21 krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450432, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:52.722994 1045 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:13:52.724359 1045 webserver.cc:460] Webserver started at http://127.4.48.194:34367/ using document root <none> and password file <none>
I20260811 12:13:52.724995 1045 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:52.725121 1045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:52.725420 1045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:52.728219 1045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "15b9ddcf2d824201ba619022c759ff41"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "6fe81fadb81dc1eb69a4a784906c72f5"
server_key_iv: "4a217bb4a8682d8d677b597aba2b9d57"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.728979 1045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "15b9ddcf2d824201ba619022c759ff41"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "6fe81fadb81dc1eb69a4a784906c72f5"
server_key_iv: "4a217bb4a8682d8d677b597aba2b9d57"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.733866 1045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 12:13:52.737291 1071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.738746 1045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:13:52.738925 1045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "15b9ddcf2d824201ba619022c759ff41"
format_stamp: "Formatted at 2026-08-11 12:13:52 on dist-test-slave-nj21"
server_key: "6fe81fadb81dc1eb69a4a784906c72f5"
server_key_iv: "4a217bb4a8682d8d677b597aba2b9d57"
server_key_version: "encryptionkey@0"
I20260811 12:13:52.739120 1045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:52.751511 1045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:52.755155 1045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:52.755421 1045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:52.756357 1045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:52.757620 1045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:52.757699 1045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.757817 1045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:52.757870 1045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:52.789078 1045 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:37197
I20260811 12:13:52.789129 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:37197 every 8 connection(s)
I20260811 12:13:52.790351 1045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:13:52.800489 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 1045
I20260811 12:13:52.800659 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:13:52.801127 4291 external_mini_cluster.cc:1428] Setting key 45c235879237ebc1438e8daeba4658df
Aug 11 12:13:52 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450432, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:52.809350 1037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:52.792761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:51723 (local address 127.4.48.254:41639)
0811 12:13:52.792944 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:52.792949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:52.794014 (+ 1065us) server_negotiation.cc:407] Connection header received
0811 12:13:52.795826 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:52.795832 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:52.795924 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:52.796012 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:52.797440 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.798148 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.799627 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.800104 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.800228 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:52.803977 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:52.803982 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:52.803986 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:52.804016 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:52.806411 (+ 2395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:52.807096 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:52.807102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:52.807106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:52.807260 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:52.807744 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:52.807748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:52.807750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:52.808020 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:52.808160 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:52.808800 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 12:13:52.809035 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:13:52.810261 1253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:52.793199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41639 (local address 127.4.48.194:51723)
0811 12:13:52.793865 (+ 666us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:52.793902 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:52.795546 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:52.796243 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:52.796251 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:52.796736 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:52.797272 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:52.797289 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.798296 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.798300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:52.799492 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:52.799504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:52.800333 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:52.800351 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:52.800554 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:52.801209 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:52.801239 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:52.803767 (+ 2528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:52.806537 (+ 2770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:52.806546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:52.806559 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:52.806960 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:52.807392 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:52.807395 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:52.807397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:52.807611 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:52.808143 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:52.808150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:52.808452 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 12:13:52.808668 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 12:13:52.809119 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":256,"threads_started":1}
I20260811 12:13:52.811650 1251 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41639
I20260811 12:13:52.811949 1251 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:52.812601 1251 heartbeater.cc:511] Master 127.4.48.254:41639 requested a full tablet report, sending...
I20260811 12:13:52.813920 716 ts_manager.cc:194] Registered new tserver with Master: 15b9ddcf2d824201ba619022c759ff41 (127.4.48.194:37197)
I20260811 12:13:52.814713 716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:51723
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:13:52.864748 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:53.188498 1260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:53.188830 1260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:53.188983 1260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:53.196070 1260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:53.196175 1260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:53.196322 1260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:13:53.205699 1260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:41639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:53.206897 1260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:53.208205 1260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:53.220786 1270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:53.220788 1271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:53.220947 1273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:53.221604 1260 server_base.cc:1034] running on GCE node
I20260811 12:13:53.222167 1260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:53.223011 1260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:53.224196 1260 hybrid_clock.cc:648] HybridClock initialized: now 1786450433224148 us; error 40 us; skew 500 ppm
I20260811 12:13:53.224489 1260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:53 dist-test-slave-nj21 krb5kdc[643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450433, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:53.227447 1260 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:13:53.228864 1260 webserver.cc:460] Webserver started at http://127.4.48.195:43703/ using document root <none> and password file <none>
I20260811 12:13:53.229512 1260 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:53.229627 1260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:53.229925 1260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:53.232738 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "68d0009775134f77a5228144e7489671"
format_stamp: "Formatted at 2026-08-11 12:13:53 on dist-test-slave-nj21"
server_key: "860c545e49a810130fb3f78a0251d02e"
server_key_iv: "49d4ba18df8f67a844a5fcbdd69dc8b4"
server_key_version: "encryptionkey@0"
I20260811 12:13:53.233556 1260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "68d0009775134f77a5228144e7489671"
format_stamp: "Formatted at 2026-08-11 12:13:53 on dist-test-slave-nj21"
server_key: "860c545e49a810130fb3f78a0251d02e"
server_key_iv: "49d4ba18df8f67a844a5fcbdd69dc8b4"
server_key_version: "encryptionkey@0"
I20260811 12:13:53.238509 1260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:13:53.241727 1286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:53.243000 1260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:13:53.243147 1260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "68d0009775134f77a5228144e7489671"
format_stamp: "Formatted at 2026-08-11 12:13:53 on dist-test-slave-nj21"
server_key: "860c545e49a810130fb3f78a0251d02e"
server_key_iv: "49d4ba18df8f67a844a5fcbdd69dc8b4"
server_key_version: "encryptionkey@0"
I20260811 12:13:53.243321 1260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:53.268824 1260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:53.272886 1260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:53.273139 1260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:53.274041 1260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:53.275312 1260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:53.275410 1260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:53.275522 1260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:53.275575 1260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:53.306365 1260 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:34677
I20260811 12:13:53.306407 1473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:34677 every 8 connection(s)
I20260811 12:13:53.307560 1260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:13:53.309051 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 1260
I20260811 12:13:53.309238 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:13:53.309763 4291 external_mini_cluster.cc:1428] Setting key ac267e7463823a392599dda0287bfa04
I20260811 12:13:53.316145 1032 heartbeater.cc:503] Master 127.4.48.254:41639 was elected leader, sending a full tablet report...
Aug 11 12:13:53 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450433, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:53.326776 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.310175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:45959 (local address 127.4.48.254:41639)
0811 12:13:53.310650 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.310656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.311716 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 12:13:53.313798 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.313803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.313866 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.313956 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:53.315310 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.316109 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.317714 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.318112 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.318205 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.321285 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:53.321291 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:53.321294 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:53.321324 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:53.323957 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.324779 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.324784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.324787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.324939 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.325417 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.325421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.325423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.325671 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:53.325781 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.326363 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.326519 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":232,"threads_started":1}
I20260811 12:13:53.327659 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.310737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:41639 (local address 127.4.48.195:45959)
0811 12:13:53.311505 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.311582 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.313551 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.314134 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.314141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.314625 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:53.315151 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.315167 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.316259 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.316264 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.317564 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.317580 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.318283 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.318287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.318497 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.319055 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:53.319076 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:53.321105 (+ 2029us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:53.324089 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:53.324098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:53.324112 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:53.324663 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:53.325058 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:53.325063 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:53.325066 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:53.325308 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:53.325766 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:53.325771 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:53.326054 (+ 283us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.326269 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.326630 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":289,"threads_started":1}
I20260811 12:13:53.328827 1476 heartbeater.cc:347] Connected to a master server at 127.4.48.254:41639
I20260811 12:13:53.329083 1476 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:53.329735 1476 heartbeater.cc:511] Master 127.4.48.254:41639 requested a full tablet report, sending...
I20260811 12:13:53.330956 716 ts_manager.cc:194] Registered new tserver with Master: 68d0009775134f77a5228144e7489671 (127.4.48.195:34677)
I20260811 12:13:53.331756 716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:45959
I20260811 12:13:53.337255 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:13:53.353539 1478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.341594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.4.48.254:41639)
0811 12:13:53.341780 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.341787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.342152 (+ 365us) server_negotiation.cc:407] Connection header received
0811 12:13:53.342301 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.342304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.342371 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.342457 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:53.343943 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.344630 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.345817 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.346220 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.346321 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.348605 (+ 2284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:53.348611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:53.348615 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:53.348647 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:53.351289 (+ 2642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.351963 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.351967 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.351969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.352089 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.352383 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.352386 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.352388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.352677 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:53.352854 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.353086 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.353248 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:13:53.357142 716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:13:53.363895 716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:13:53.385516 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.376042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39526 (local address 127.4.48.195:34677)
0811 12:13:53.377465 (+ 1423us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.377472 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.377509 (+ 37us) server_negotiation.cc:407] Connection header received
0811 12:13:53.377613 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.377621 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.377855 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.378068 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:53.378917 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.380560 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.383054 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.384365 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.384487 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.384720 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.385011 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.385163 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1240,"spinlock_wait_cycles":19200,"thread_start_us":1209,"threads_started":1}
I20260811 12:13:53.385588 1496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.375393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:34677 (local address 127.0.0.1:39526)
0811 12:13:53.377347 (+ 1954us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.377366 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.377485 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.378076 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.378083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.378115 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:53.378642 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.378660 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.380764 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.380769 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.382903 (+ 2134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.382916 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.384557 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.384564 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.384769 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.384822 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.384942 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.385113 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"mutex_wait_us":35,"thread_start_us":2383,"threads_started":1}
I20260811 12:13:53.386083 1495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.375265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:37197 (local address 127.0.0.1:38912)
0811 12:13:53.376571 (+ 1306us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.376611 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.376821 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.378527 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.378531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.378561 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:53.379041 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.379055 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.381046 (+ 1991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.381051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.383346 (+ 2295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.383361 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.385252 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.385259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.385430 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.385466 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.385532 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.385671 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":977,"mutex_wait_us":2,"thread_start_us":840,"threads_started":1}
I20260811 12:13:53.386897 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.375446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38912 (local address 127.4.48.194:37197)
0811 12:13:53.377963 (+ 2517us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.377969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.377992 (+ 23us) server_negotiation.cc:407] Connection header received
0811 12:13:53.378078 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.378084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.378311 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.379234 (+ 923us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:53.379460 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.380929 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.383547 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.385011 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.386165 (+ 1154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.386349 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.386414 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.386569 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2383,"thread_start_us":2249,"threads_started":1}
I20260811 12:13:53.388155 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.378116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34601 (local address 127.0.0.1:42412)
0811 12:13:53.378930 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.378949 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.379066 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.380781 (+ 1715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.380786 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.380818 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:53.381454 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.381472 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.383571 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.383576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.385709 (+ 2133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.385722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.387334 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.387340 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.387538 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.387571 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.387660 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.387803 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"mutex_wait_us":21,"thread_start_us":789,"threads_started":1}
I20260811 12:13:53.388521 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.378724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42412 (local address 127.4.48.193:34601)
0811 12:13:53.380199 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.380206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.380233 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:13:53.380329 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.380337 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.380573 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.381215 (+ 642us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:53.381674 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.383399 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.385893 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.387111 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.387244 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.387462 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.387945 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.388096 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1284,"thread_start_us":2050,"threads_started":1}
I20260811 12:13:53.390167 1369 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (DEFAULT_TABLE table=test-table [id=55101d75e503479b9f21f908029439bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:53.390290 1142 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (DEFAULT_TABLE table=test-table [id=55101d75e503479b9f21f908029439bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:53.391709 1142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ed2f985c8db4ace8d0f0f6aa741b174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:53.391708 1369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ed2f985c8db4ace8d0f0f6aa741b174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:53.391376 923 tablet_service.cc:1467] Processing CreateTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (DEFAULT_TABLE table=test-table [id=55101d75e503479b9f21f908029439bd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:53.392493 923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ed2f985c8db4ace8d0f0f6aa741b174. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:53.403228 1504 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Bootstrap starting.
I20260811 12:13:53.405738 1505 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Bootstrap starting.
I20260811 12:13:53.406711 1504 tablet_bootstrap.cc:654] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:53.407831 1506 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Bootstrap starting.
I20260811 12:13:53.408396 1504 log.cc:824] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:53.409510 1505 tablet_bootstrap.cc:654] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:53.410660 1505 log.cc:824] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:53.410862 1506 tablet_bootstrap.cc:654] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:53.411545 1504 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: No bootstrap required, opened a new log
I20260811 12:13:53.411803 1504 ts_tablet_manager.cc:1397] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:13:53.412072 1506 log.cc:824] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:53.413640 1505 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: No bootstrap required, opened a new log
I20260811 12:13:53.413884 1505 ts_tablet_manager.cc:1397] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 12:13:53.414423 1506 tablet_bootstrap.cc:492] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: No bootstrap required, opened a new log
I20260811 12:13:53.414655 1506 ts_tablet_manager.cc:1397] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 12:13:53.421787 1504 raft_consensus.cc:348] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.421983 1506 raft_consensus.cc:348] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.422209 1504 raft_consensus.cc:374] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:53.422299 1504 raft_consensus.cc:729] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 235bcce3d1b34292a0dad338b9138c3c, State: Initialized, Role: FOLLOWER
I20260811 12:13:53.422320 1506 raft_consensus.cc:374] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:53.422418 1506 raft_consensus.cc:729] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68d0009775134f77a5228144e7489671, State: Initialized, Role: FOLLOWER
I20260811 12:13:53.422966 1506 consensus_queue.cc:260] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.423051 1504 consensus_queue.cc:260] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.423733 1505 raft_consensus.cc:348] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.424119 1505 raft_consensus.cc:374] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:53.424206 1505 raft_consensus.cc:729] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15b9ddcf2d824201ba619022c759ff41, State: Initialized, Role: FOLLOWER
I20260811 12:13:53.424829 1504 ts_tablet_manager.cc:1428] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Time spent starting tablet: real 0.013s user 0.008s sys 0.004s
I20260811 12:13:53.424835 1505 consensus_queue.cc:260] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.425726 1476 heartbeater.cc:503] Master 127.4.48.254:41639 was elected leader, sending a full tablet report...
I20260811 12:13:53.426322 1506 ts_tablet_manager.cc:1428] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 12:13:53.427510 1251 heartbeater.cc:503] Master 127.4.48.254:41639 was elected leader, sending a full tablet report...
I20260811 12:13:53.429013 1505 ts_tablet_manager.cc:1428] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 12:13:53.535535 1511 raft_consensus.cc:482] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:53.535856 1511 raft_consensus.cc:504] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
W20260811 12:13:53.537326 1033 tablet.cc:2369] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:13:53.537601 1511 leader_election.cc:288] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15b9ddcf2d824201ba619022c759ff41 (127.4.48.194:37197), 235bcce3d1b34292a0dad338b9138c3c (127.4.48.193:34601)
W20260811 12:13:53.542207 1252 tablet.cc:2369] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:13:53.543830 1479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.537785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:37197 (local address 127.4.48.195:37391)
0811 12:13:53.538072 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.538093 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.538240 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.538777 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.538782 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.538813 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:53.539017 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.539030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.540361 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.540367 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.541917 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.541929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.543197 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.543203 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.543356 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.543391 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.543465 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.543554 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":47}
I20260811 12:13:53.543936 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.538223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:37391 (local address 127.4.48.194:37197)
0811 12:13:53.538431 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.538437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.538457 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:13:53.538531 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.538537 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.538617 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.538713 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:53.539185 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.540215 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.542069 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.543046 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.543186 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.543259 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.543514 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.543648 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:13:53.544864 1519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.538062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34601 (local address 127.4.48.195:56297)
0811 12:13:53.538918 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:53.538932 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:53.539066 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:53.539335 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:53.539339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:53.539363 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:53.539816 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.539830 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.541074 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.541078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.542741 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:53.542756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.544311 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.544315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:53.544431 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.544454 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:13:53.544528 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 12:13:53.544606 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"mutex_wait_us":17,"thread_start_us":598,"threads_started":1}
I20260811 12:13:53.544939 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.538501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:56297 (local address 127.4.48.193:34601)
0811 12:13:53.538659 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.538663 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.539040 (+ 377us) server_negotiation.cc:407] Connection header received
0811 12:13:53.539174 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.539178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.539237 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.539304 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:53.539975 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.540917 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.542941 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.544102 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.544247 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.544311 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.544581 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.544707 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 12:13:53.544880 1178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ed2f985c8db4ace8d0f0f6aa741b174" candidate_uuid: "68d0009775134f77a5228144e7489671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b9ddcf2d824201ba619022c759ff41" is_pre_election: true
I20260811 12:13:53.545477 1178 raft_consensus.cc:2454] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d0009775134f77a5228144e7489671 in term 0.
I20260811 12:13:53.545951 958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ed2f985c8db4ace8d0f0f6aa741b174" candidate_uuid: "68d0009775134f77a5228144e7489671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235bcce3d1b34292a0dad338b9138c3c" is_pre_election: true
I20260811 12:13:53.546164 1287 leader_election.cc:302] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15b9ddcf2d824201ba619022c759ff41, 68d0009775134f77a5228144e7489671; no voters:
I20260811 12:13:53.546519 958 raft_consensus.cc:2454] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 68d0009775134f77a5228144e7489671 in term 0.
I20260811 12:13:53.546581 1511 raft_consensus.cc:2781] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:13:53.546703 1511 raft_consensus.cc:482] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:13:53.546797 1511 raft_consensus.cc:3037] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:53.549096 1511 raft_consensus.cc:504] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.549624 1511 leader_election.cc:288] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [CANDIDATE]: Term 1 election: Requested vote from peers 15b9ddcf2d824201ba619022c759ff41 (127.4.48.194:37197), 235bcce3d1b34292a0dad338b9138c3c (127.4.48.193:34601)
I20260811 12:13:53.550058 958 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ed2f985c8db4ace8d0f0f6aa741b174" candidate_uuid: "68d0009775134f77a5228144e7489671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "235bcce3d1b34292a0dad338b9138c3c"
I20260811 12:13:53.550091 1178 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ed2f985c8db4ace8d0f0f6aa741b174" candidate_uuid: "68d0009775134f77a5228144e7489671" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15b9ddcf2d824201ba619022c759ff41"
I20260811 12:13:53.550230 958 raft_consensus.cc:3037] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:53.550251 1178 raft_consensus.cc:3037] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:53.552395 1178 raft_consensus.cc:2454] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d0009775134f77a5228144e7489671 in term 1.
I20260811 12:13:53.552448 958 raft_consensus.cc:2454] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 68d0009775134f77a5228144e7489671 in term 1.
I20260811 12:13:53.552752 1287 leader_election.cc:302] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15b9ddcf2d824201ba619022c759ff41, 68d0009775134f77a5228144e7489671; no voters:
I20260811 12:13:53.553038 1511 raft_consensus.cc:2781] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:53.553452 1511 raft_consensus.cc:686] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 1 LEADER]: Becoming Leader. State: Replica: 68d0009775134f77a5228144e7489671, State: Running, Role: LEADER
I20260811 12:13:53.553858 1511 consensus_queue.cc:237] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } }
I20260811 12:13:53.558033 714 catalog_manager.cc:5495] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 reported cstate change: term changed from 0 to 1, leader changed from <none> to 68d0009775134f77a5228144e7489671 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "68d0009775134f77a5228144e7489671" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "68d0009775134f77a5228144e7489671" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 34677 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 } health_report { overall_health: UNKNOWN } } }
W20260811 12:13:53.558977 1477 tablet.cc:2369] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:13:53.577842 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.572895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39542 (local address 127.4.48.195:34677)
0811 12:13:53.573084 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.573089 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.573227 (+ 138us) server_negotiation.cc:407] Connection header received
0811 12:13:53.573424 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.573428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.573498 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.573615 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:13:53.573937 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.574597 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.575943 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.576326 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.576420 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.576864 (+ 444us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:13:53.577397 (+ 533us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:13:53.577510 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.577587 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.577648 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:13:53.585213 958 raft_consensus.cc:1261] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 1 FOLLOWER]: Refusing update from remote peer 68d0009775134f77a5228144e7489671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:53.585234 1178 raft_consensus.cc:1261] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 1 FOLLOWER]: Refusing update from remote peer 68d0009775134f77a5228144e7489671: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:53.586109 1525 consensus_queue.cc:1035] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "235bcce3d1b34292a0dad338b9138c3c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:53.586445 1511 consensus_queue.cc:1035] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15b9ddcf2d824201ba619022c759ff41" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:53.596771 1532 mvcc.cc:205] Tried to move back new op lower bound from 7317300975956017152 to 7317300975840088064. Current Snapshot: MvccSnapshot[applied={T|T < 7317300975956017152}]
I20260811 12:13:53.598623 1534 mvcc.cc:205] Tried to move back new op lower bound from 7317300975956017152 to 7317300975840088064. Current Snapshot: MvccSnapshot[applied={T|T < 7317300975956017152}]
I20260811 12:13:53.601178 1535 mvcc.cc:205] Tried to move back new op lower bound from 7317300975956017152 to 7317300975840088064. Current Snapshot: MvccSnapshot[applied={T|T < 7317300975956017152}]
I20260811 12:13:53.609380 714 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:53.609633 714 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:53.613163 714 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9bd02f63051411e83c3272815898878: Sending DeleteTablet for 3 replicas of tablet 4ed2f985c8db4ace8d0f0f6aa741b174
I20260811 12:13:53.614341 1142 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:53 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:38912
I20260811 12:13:53.614733 1369 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:53 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:39526
I20260811 12:13:53.615437 923 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ed2f985c8db4ace8d0f0f6aa741b174 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:53 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:42412
I20260811 12:13:53.615780 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 809
I20260811 12:13:53.619241 1555 tablet_replica.cc:330] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: stopping tablet replica
I20260811 12:13:53.619735 1555 raft_consensus.cc:2229] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:53.620239 1555 raft_consensus.cc:2258] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:53.621083 1554 tablet_replica.cc:330] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: stopping tablet replica
I20260811 12:13:53.621487 1554 raft_consensus.cc:2229] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:13:53.622145 1554 raft_consensus.cc:2258] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:53.623324 1551 tablet_replica.cc:330] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: stopping tablet replica
I20260811 12:13:53.623740 1551 raft_consensus.cc:2229] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:53.624222 1551 raft_consensus.cc:2258] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:53.625794 1555 ts_tablet_manager.cc:1905] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:53.625789 1554 ts_tablet_manager.cc:1905] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:53.627063 1551 ts_tablet_manager.cc:1905] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 12:13:53 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:13:53.633285 1554 ts_tablet_manager.cc:1918] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:53.633546 1554 log.cc:1196] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/4ed2f985c8db4ace8d0f0f6aa741b174
I20260811 12:13:53.634137 1554 ts_tablet_manager.cc:1939] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 68d0009775134f77a5228144e7489671: Deleting consensus metadata
I20260811 12:13:53.636116 692 catalog_manager.cc:4841] TS 68d0009775134f77a5228144e7489671 (127.4.48.195:34677): tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (table test-table [id=55101d75e503479b9f21f908029439bd]) successfully deleted
I20260811 12:13:53.637845 1551 ts_tablet_manager.cc:1918] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:53.638032 1551 log.cc:1196] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/4ed2f985c8db4ace8d0f0f6aa741b174
I20260811 12:13:53.638528 1551 ts_tablet_manager.cc:1939] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 15b9ddcf2d824201ba619022c759ff41: Deleting consensus metadata
I20260811 12:13:53.638581 1555 ts_tablet_manager.cc:1918] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:53.638763 1555 log.cc:1196] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/4ed2f985c8db4ace8d0f0f6aa741b174
I20260811 12:13:53.639259 1555 ts_tablet_manager.cc:1939] T 4ed2f985c8db4ace8d0f0f6aa741b174 P 235bcce3d1b34292a0dad338b9138c3c: Deleting consensus metadata
I20260811 12:13:53.640282 687 catalog_manager.cc:4841] TS 15b9ddcf2d824201ba619022c759ff41 (127.4.48.194:37197): tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (table test-table [id=55101d75e503479b9f21f908029439bd]) successfully deleted
I20260811 12:13:53.641171 692 catalog_manager.cc:4841] TS 235bcce3d1b34292a0dad338b9138c3c (127.4.48.193:34601): tablet 4ed2f985c8db4ace8d0f0f6aa741b174 (table test-table [id=55101d75e503479b9f21f908029439bd]) successfully deleted
I20260811 12:13:53.645195 1503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.616966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42426 (local address 127.4.48.193:34601)
0811 12:13:53.617145 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.617151 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.618595 (+ 1444us) server_negotiation.cc:407] Connection header received
0811 12:13:53.618781 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.618787 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.618927 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.619052 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:53.622692 (+ 3640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.623835 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.625735 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.626312 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.626411 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.631940 (+ 5529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:53.631955 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:53.631972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:53.632012 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:53.641778 (+ 9766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.642668 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.642677 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.642682 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.642848 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.643281 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.643286 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.643290 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.643780 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:53.644049 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.644427 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.644636 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:13:53.645835 1021 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:53.740214 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 809
I20260811 12:13:53.768132 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 1045
Aug 11 12:13:53 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:13:53.784917 1498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.769071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38916 (local address 127.4.48.194:37197)
0811 12:13:53.769293 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.769300 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.769347 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:13:53.769428 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.769435 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.769538 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.769655 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:53.770877 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.772037 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.773325 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.773957 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.774077 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.779035 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:53.779060 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:53.779084 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:53.779164 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:53.781979 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.782824 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.782833 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.782835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.782962 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.783362 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.783366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.783368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.783920 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:53.784154 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.784458 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.784620 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:13:53.785681 1244 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:53.873117 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 1045
I20260811 12:13:53.905478 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 1260
Aug 11 12:13:53 dist-test-slave-nj21 krb5kdc[643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450431, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:13:53.921144 1501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:53.906394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39548 (local address 127.4.48.195:34677)
0811 12:13:53.906589 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:53.906595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:53.906613 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:13:53.906672 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:53.906676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:53.906762 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:53.906858 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:53.908245 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.909100 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.910379 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:53.910803 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:53.910918 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:53.915419 (+ 4501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:53.915436 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:53.915455 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:53.915488 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:53.918322 (+ 2834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.918966 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.918974 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.918976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.919130 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:53.919522 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:53.919525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:53.919527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:53.919932 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:53.920248 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:53.920569 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 12:13:53.920776 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:13:53.921940 1462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:54.041179 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 1260
I20260811 12:13:54.067538 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 662
I20260811 12:13:54.068720 764 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:54.188457 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 662
2026-08-11T12:13:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4490 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T12:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:13:54Z Disabled control of system clock
I20260811 12:13:54.282938 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45507
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:40637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:13:54.613407 1569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:54.613677 1569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:54.613766 1569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:54.620976 1569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:13:54.621068 1569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:54.621112 1569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:54.621166 1569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:13:54.621208 1569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:13:54.630072 1569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45507
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:54.631269 1569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:54.633062 1569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:54.644156 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:54.644085 1576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:54.644131 1578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:54.645083 1569 server_base.cc:1034] running on GCE node
I20260811 12:13:54.645787 1569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:54.647099 1569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:54.648293 1569 hybrid_clock.cc:648] HybridClock initialized: now 1786450434648237 us; error 49 us; skew 500 ppm
I20260811 12:13:54.648619 1569 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:13:54.656657 1569 webserver.cc:460] Webserver started at http://127.4.48.254:33623/ using document root <none> and password file <none>
I20260811 12:13:54.657318 1569 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:54.657426 1569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:54.657730 1569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:54.660573 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "a51f31b78b3d4b08afb00af44b887bdd"
format_stamp: "Formatted at 2026-08-11 12:13:54 on dist-test-slave-nj21"
server_key: "eb353769573d430d6f892be8d0ed4dfd"
server_key_iv: "3790c45dda0b3d1bf49bf504c34039f6"
server_key_version: "encryptionkey@0"
I20260811 12:13:54.661321 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "a51f31b78b3d4b08afb00af44b887bdd"
format_stamp: "Formatted at 2026-08-11 12:13:54 on dist-test-slave-nj21"
server_key: "eb353769573d430d6f892be8d0ed4dfd"
server_key_iv: "3790c45dda0b3d1bf49bf504c34039f6"
server_key_version: "encryptionkey@0"
I20260811 12:13:54.666126 1569 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:13:54.669330 1588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:54.670567 1569 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:13:54.670728 1569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "a51f31b78b3d4b08afb00af44b887bdd"
format_stamp: "Formatted at 2026-08-11 12:13:54 on dist-test-slave-nj21"
server_key: "eb353769573d430d6f892be8d0ed4dfd"
server_key_iv: "3790c45dda0b3d1bf49bf504c34039f6"
server_key_version: "encryptionkey@0"
I20260811 12:13:54.670913 1569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:54.684585 1569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:54.685274 1569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:54.685528 1569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:54.702286 1569 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45507
I20260811 12:13:54.702304 1673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45507 every 8 connection(s)
I20260811 12:13:54.703496 1569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:13:54.707070 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 1569
I20260811 12:13:54.707209 1675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:54.707245 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:13:54.707896 4291 external_mini_cluster.cc:1428] Setting key c11f1d437d17692745a301c2fac767d7
I20260811 12:13:54.719837 1675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: Bootstrap starting.
I20260811 12:13:54.720563 1682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:54.709734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.4.48.254:45507)
0811 12:13:54.710715 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:54.710728 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:54.710770 (+ 42us) server_negotiation.cc:407] Connection header received
0811 12:13:54.712334 (+ 1564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:54.712355 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:54.712837 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:54.713181 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:54.713735 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:54.715007 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:54.716398 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:54.717078 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:54.717254 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:54.718378 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:54.718385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:54.718398 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:54.718889 (+ 491us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:54.718960 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:54.718979 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:54.719076 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:54.719299 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 12:13:54.719570 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":363,"threads_started":1}
I20260811 12:13:54.722867 1675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:54.723922 1675 log.cc:824] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:54.726603 1675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: No bootstrap required, opened a new log
I20260811 12:13:54.734254 1675 raft_consensus.cc:348] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } }
I20260811 12:13:54.734580 1675 raft_consensus.cc:374] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:54.734689 1675 raft_consensus.cc:729] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a51f31b78b3d4b08afb00af44b887bdd, State: Initialized, Role: FOLLOWER
I20260811 12:13:54.735242 1675 consensus_queue.cc:260] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } }
I20260811 12:13:54.735466 1675 raft_consensus.cc:388] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:13:54.735591 1675 raft_consensus.cc:482] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:13:54.735777 1675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:54.737984 1675 raft_consensus.cc:504] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } }
I20260811 12:13:54.738507 1675 leader_election.cc:302] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a51f31b78b3d4b08afb00af44b887bdd; no voters:
I20260811 12:13:54.739076 1675 leader_election.cc:288] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:13:54.739161 1685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:54.739853 1685 raft_consensus.cc:686] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [term 1 LEADER]: Becoming Leader. State: Replica: a51f31b78b3d4b08afb00af44b887bdd, State: Running, Role: LEADER
I20260811 12:13:54.740388 1685 consensus_queue.cc:237] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } }
I20260811 12:13:54.741008 1675 sys_catalog.cc:564] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:13:54.742935 1686 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a51f31b78b3d4b08afb00af44b887bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } } }
I20260811 12:13:54.743527 1686 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:54.743072 1687 sys_catalog.cc:455] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a51f31b78b3d4b08afb00af44b887bdd. Latest consensus state: current_term: 1 leader_uuid: "a51f31b78b3d4b08afb00af44b887bdd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a51f31b78b3d4b08afb00af44b887bdd" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45507 } } }
I20260811 12:13:54.743637 1687 sys_catalog.cc:458] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:54.755239 1692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:13:54.760272 1692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:13:54.768607 1692 catalog_manager.cc:1347] Generated new cluster ID: 9a56a21a79764812bb2c8ecec7040b58
I20260811 12:13:54.768733 1692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:13:54.815593 1692 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:13:54.817430 1692 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:13:54.825353 1692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: Generated new TSK 0
I20260811 12:13:54.826378 1692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:13:54.838603 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--builtin_ntp_servers=127.4.48.212:40637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:13:55.177223 1714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:55.177503 1714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:55.177630 1714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:55.184588 1714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:55.184686 1714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:55.184837 1714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:13:55.193650 1714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:55.194902 1714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:55.196725 1714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:55.209369 1720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:55.209381 1721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:55.209590 1725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:55.210182 1714 server_base.cc:1034] running on GCE node
I20260811 12:13:55.210759 1714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:55.211655 1714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:55.212922 1714 hybrid_clock.cc:648] HybridClock initialized: now 1786450435212852 us; error 47 us; skew 500 ppm
I20260811 12:13:55.213224 1714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:13:55.215503 1714 webserver.cc:460] Webserver started at http://127.4.48.193:38655/ using document root <none> and password file <none>
I20260811 12:13:55.216213 1714 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:55.216316 1714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:55.216626 1714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:55.219380 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "80446c50de5d4948af37656460582e1c"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "f4fee950a6b4b8f7c78f204051aad76a"
server_key_iv: "0c14e686f16b8b50c4994ad35690f0bf"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.220218 1714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "80446c50de5d4948af37656460582e1c"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "f4fee950a6b4b8f7c78f204051aad76a"
server_key_iv: "0c14e686f16b8b50c4994ad35690f0bf"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.225077 1714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:13:55.228509 1735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.229797 1714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 12:13:55.229959 1714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "80446c50de5d4948af37656460582e1c"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "f4fee950a6b4b8f7c78f204051aad76a"
server_key_iv: "0c14e686f16b8b50c4994ad35690f0bf"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.230150 1714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:55.255268 1714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:55.256100 1714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:55.256388 1714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:55.257366 1714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:55.258845 1714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:55.258953 1714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.259058 1714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:55.259133 1714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.288379 1714 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:36141
I20260811 12:13:55.288457 1920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:36141 every 8 connection(s)
I20260811 12:13:55.289649 1714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:13:55.298813 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 1714
I20260811 12:13:55.299008 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:13:55.299726 4291 external_mini_cluster.cc:1428] Setting key ded4c37a8c9e92ddeda50a6a7b80fd40
I20260811 12:13:55.302836 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:55.292294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:33453 (local address 127.4.48.254:45507)
0811 12:13:55.292784 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:55.292789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:55.293771 (+ 982us) server_negotiation.cc:407] Connection header received
0811 12:13:55.295652 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:55.295658 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:55.295772 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:55.295882 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:55.296987 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.297759 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.299942 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.300344 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.300450 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:55.301941 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:55.301949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:55.301954 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:55.301977 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:55.302032 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:55.302052 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:55.302155 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:55.302345 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 12:13:55.302447 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":239,"threads_started":1}
I20260811 12:13:55.302924 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--builtin_ntp_servers=127.4.48.212:40637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:13:55.303524 1927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:55.292702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45507 (local address 127.4.48.193:33453)
0811 12:13:55.293598 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:55.293635 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:55.295375 (+ 1740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:55.296120 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:55.296129 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:55.296230 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:55.296810 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:55.296828 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.297924 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.297928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:55.299799 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:55.299814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.300558 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.300564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:55.300808 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:55.301570 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:13:55.301573 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:55.301631 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:13:55.301727 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:55.302152 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:55.302158 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:55.302164 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:13:55.302296 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 12:13:55.302561 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":276,"threads_started":1}
I20260811 12:13:55.305068 1922 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45507
I20260811 12:13:55.305333 1922 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:55.306010 1922 heartbeater.cc:511] Master 127.4.48.254:45507 requested a full tablet report, sending...
I20260811 12:13:55.307981 1615 ts_manager.cc:194] Registered new tserver with Master: 80446c50de5d4948af37656460582e1c (127.4.48.193:36141)
I20260811 12:13:55.309468 1615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.193:33453
W20260811 12:13:55.629804 1929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:55.630056 1929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:55.630148 1929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:55.637154 1929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:55.637238 1929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:55.637342 1929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:13:55.646227 1929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:55.647414 1929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:55.649143 1929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:55.661533 1935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:55.661572 1941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:55.661605 1936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:55.662387 1929 server_base.cc:1034] running on GCE node
I20260811 12:13:55.662972 1929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:55.663908 1929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:55.665125 1929 hybrid_clock.cc:648] HybridClock initialized: now 1786450435665059 us; error 47 us; skew 500 ppm
I20260811 12:13:55.665422 1929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:13:55.667549 1929 webserver.cc:460] Webserver started at http://127.4.48.194:42257/ using document root <none> and password file <none>
I20260811 12:13:55.668213 1929 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:55.668319 1929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:55.668639 1929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:55.671437 1929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "6b1d24fe92e74333843374cebbf19628"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "0aa31fb82b97629ded717d74f0970d03"
server_key_iv: "354c932d86a3af86ba64bbd32ef6d05a"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.672245 1929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "6b1d24fe92e74333843374cebbf19628"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "0aa31fb82b97629ded717d74f0970d03"
server_key_iv: "354c932d86a3af86ba64bbd32ef6d05a"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.677953 1929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:13:55.682042 1955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.683395 1929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:13:55.683545 1929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "6b1d24fe92e74333843374cebbf19628"
format_stamp: "Formatted at 2026-08-11 12:13:55 on dist-test-slave-nj21"
server_key: "0aa31fb82b97629ded717d74f0970d03"
server_key_iv: "354c932d86a3af86ba64bbd32ef6d05a"
server_key_version: "encryptionkey@0"
I20260811 12:13:55.683789 1929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:55.702592 1929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:55.703336 1929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:55.703614 1929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:55.704559 1929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:55.706065 1929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:55.706161 1929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.706267 1929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:55.706339 1929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:55.736984 1929 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:45311
I20260811 12:13:55.737057 2154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:45311 every 8 connection(s)
I20260811 12:13:55.738200 1929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:13:55.741271 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 1929
I20260811 12:13:55.741446 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:13:55.742049 4291 external_mini_cluster.cc:1428] Setting key 2089359201bd48b7c75b575edabd2729
I20260811 12:13:55.745182 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--builtin_ntp_servers=127.4.48.212:40637
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:13:55.751226 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:55.740642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:39665 (local address 127.4.48.254:45507)
0811 12:13:55.740842 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:55.740849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:55.742463 (+ 1614us) server_negotiation.cc:407] Connection header received
0811 12:13:55.744554 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:55.744558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:55.744623 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:55.744717 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:55.746012 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.746695 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.748734 (+ 2039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.749124 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.749212 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:55.750585 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:55.750590 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:55.750593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:55.750610 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:55.750641 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:55.750654 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:55.750718 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:55.750927 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 12:13:55.750999 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:13:55.752529 2157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:55.741205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45507 (local address 127.4.48.194:39665)
0811 12:13:55.742313 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:55.742360 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:55.744271 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:55.744970 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:55.744981 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:55.745091 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:55.745857 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:55.745881 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.746852 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.746858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:55.748620 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:55.748633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:55.749326 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:55.749333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:55.749600 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:55.750266 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:13:55.750269 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:55.750316 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:13:55.750410 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:55.750745 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:55.750756 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:55.750764 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:13:55.750926 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 12:13:55.751282 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":324,"threads_started":1}
I20260811 12:13:55.753829 2155 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45507
I20260811 12:13:55.754112 2155 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:55.754822 2155 heartbeater.cc:511] Master 127.4.48.254:45507 requested a full tablet report, sending...
I20260811 12:13:55.756237 1615 ts_manager.cc:194] Registered new tserver with Master: 6b1d24fe92e74333843374cebbf19628 (127.4.48.194:45311)
I20260811 12:13:55.757208 1615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.194:39665
W20260811 12:13:56.076452 2159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:56.076711 2159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:56.076843 2159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:56.083685 2159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:56.083786 2159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:56.083921 2159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:13:56.092566 2159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40637
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:56.093739 2159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:56.095436 2159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:56.107544 2166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:56.107586 2169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:56.107502 2171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:56.108820 2159 server_base.cc:1034] running on GCE node
I20260811 12:13:56.109364 2159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:56.110224 2159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:56.111436 2159 hybrid_clock.cc:648] HybridClock initialized: now 1786450436111383 us; error 38 us; skew 500 ppm
I20260811 12:13:56.111783 2159 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:13:56.113854 2159 webserver.cc:460] Webserver started at http://127.4.48.195:36041/ using document root <none> and password file <none>
I20260811 12:13:56.114509 2159 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:56.114611 2159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:56.114971 2159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:56.117717 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "999009e3083348849f4df5a4ba101f73"
format_stamp: "Formatted at 2026-08-11 12:13:56 on dist-test-slave-nj21"
server_key: "442a76c6389438c4d59218f34236f4eb"
server_key_iv: "c9f5bc604a6aed43f387da1522e29060"
server_key_version: "encryptionkey@0"
I20260811 12:13:56.118471 2159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "999009e3083348849f4df5a4ba101f73"
format_stamp: "Formatted at 2026-08-11 12:13:56 on dist-test-slave-nj21"
server_key: "442a76c6389438c4d59218f34236f4eb"
server_key_iv: "c9f5bc604a6aed43f387da1522e29060"
server_key_version: "encryptionkey@0"
I20260811 12:13:56.123167 2159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 12:13:56.126444 2184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:56.127665 2159 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:13:56.127849 2159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "999009e3083348849f4df5a4ba101f73"
format_stamp: "Formatted at 2026-08-11 12:13:56 on dist-test-slave-nj21"
server_key: "442a76c6389438c4d59218f34236f4eb"
server_key_iv: "c9f5bc604a6aed43f387da1522e29060"
server_key_version: "encryptionkey@0"
I20260811 12:13:56.128057 2159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:56.140842 2159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:56.141619 2159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:56.141899 2159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:56.142805 2159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:56.144205 2159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:56.144304 2159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:56.144410 2159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:56.144479 2159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:56.174057 2159 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:40545
I20260811 12:13:56.174134 2373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:40545 every 8 connection(s)
I20260811 12:13:56.175418 2159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:13:56.184125 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 2159
I20260811 12:13:56.184290 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:13:56.184703 4291 external_mini_cluster.cc:1428] Setting key 6e005cec12be12eeffb832d9681cdec1
I20260811 12:13:56.187450 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.177781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:35417 (local address 127.4.48.254:45507)
0811 12:13:56.177953 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.177958 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.179257 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 12:13:56.181047 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.181052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.181137 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.181230 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:56.182270 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.183013 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.184624 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.185092 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.185228 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.186624 (+ 1396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:56.186630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:56.186635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:56.186658 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:56.186698 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:56.186714 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:56.186804 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.187018 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.187117 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:13:56.188443 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.178264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45507 (local address 127.4.48.195:35417)
0811 12:13:56.179103 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.179148 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.180777 (+ 1629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.181477 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.181486 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.181569 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:56.182104 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.182121 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.183165 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.183169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.184484 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.184497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.185306 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.185314 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.185600 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.186265 (+ 665us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:13:56.186268 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:56.186318 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:13:56.186425 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:56.186818 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:56.186829 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:56.186837 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.186983 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.187303 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":261,"threads_started":1}
I20260811 12:13:56.189585 2374 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45507
I20260811 12:13:56.189817 2374 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:56.190552 2374 heartbeater.cc:511] Master 127.4.48.254:45507 requested a full tablet report, sending...
I20260811 12:13:56.191804 1615 ts_manager.cc:194] Registered new tserver with Master: 999009e3083348849f4df5a4ba101f73 (127.4.48.195:40545)
I20260811 12:13:56.192576 1615 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.195:35417
I20260811 12:13:56.199664 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:13:56.208683 1924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.203532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55318 (local address 127.4.48.254:45507)
0811 12:13:56.203728 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.203734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.204109 (+ 375us) server_negotiation.cc:407] Connection header received
0811 12:13:56.204227 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.204231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.204288 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.204363 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:56.204779 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.205460 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.206751 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.207139 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.207230 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.208148 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:56.208153 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:56.208156 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:56.208173 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:56.208203 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:56.208215 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:56.208282 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.208356 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.208428 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 12:13:56.211992 1615 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:13:56.218650 1615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:13:56.238837 2396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.229996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36141 (local address 127.0.0.1:49458)
0811 12:13:56.230717 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.230749 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.230875 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.231936 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.231945 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.231983 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:56.232309 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.232321 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.234105 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.234111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.236301 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.236316 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.238096 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.238101 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.238278 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.238322 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.238415 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.238544 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":19,"thread_start_us":307,"threads_started":1}
I20260811 12:13:56.240329 2400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.231067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45311 (local address 127.0.0.1:42670)
0811 12:13:56.231572 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.231588 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.231713 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.233553 (+ 1840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.233558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.233583 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:56.234058 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.234075 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.235969 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.235975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.237863 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.237879 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.239619 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.239625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.239813 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.239845 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.239921 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.240044 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":244,"threads_started":1}
I20260811 12:13:56.240509 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.231472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.4.48.194:45311)
0811 12:13:56.232434 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.232442 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.232472 (+ 30us) server_negotiation.cc:407] Connection header received
0811 12:13:56.232571 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.232578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.232795 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.233549 (+ 754us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:56.234268 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.235826 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.238051 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.239429 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.239563 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.239872 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.239961 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.240131 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":663,"threads_started":1}
I20260811 12:13:56.241509 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.230083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49458 (local address 127.4.48.193:36141)
0811 12:13:56.230771 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.230779 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.230831 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:13:56.231075 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.231081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.231401 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.231585 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:56.232470 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.233908 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.236475 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.237882 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.239359 (+ 1477us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.240936 (+ 1577us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.241007 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.241165 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":409,"threads_started":1}
I20260811 12:13:56.244401 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.231639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:40545 (local address 127.0.0.1:35462)
0811 12:13:56.233116 (+ 1477us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.233131 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.233360 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.233928 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.233933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.233957 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:56.236586 (+ 2629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.236596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.237964 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.237969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.240041 (+ 2072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.240053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.242175 (+ 2122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.242183 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.243859 (+ 1676us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.243891 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.243968 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.244080 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1368,"mutex_wait_us":7,"thread_start_us":863,"threads_started":1}
I20260811 12:13:56.244519 1826 tablet_service.cc:1467] Processing CreateTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 (DEFAULT_TABLE table=test-table [id=291fd70e58b2462b9409de41043b78a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:56.244339 2046 tablet_service.cc:1467] Processing CreateTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 (DEFAULT_TABLE table=test-table [id=291fd70e58b2462b9409de41043b78a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:56.245230 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.232218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.4.48.195:40545)
0811 12:13:56.232935 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.232944 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.233261 (+ 317us) server_negotiation.cc:407] Connection header received
0811 12:13:56.233532 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.233541 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.233743 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.233928 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:56.236743 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.237822 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.240250 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.241971 (+ 1721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.242104 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.242426 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.244672 (+ 2246us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.244882 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":911,"threads_started":1}
I20260811 12:13:56.245874 2046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556f5aba44534a65a2ff9b4bec90c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:56.245903 1826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556f5aba44534a65a2ff9b4bec90c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:56.248945 2263 tablet_service.cc:1467] Processing CreateTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 (DEFAULT_TABLE table=test-table [id=291fd70e58b2462b9409de41043b78a2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:13:56.250427 2263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 556f5aba44534a65a2ff9b4bec90c857. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:56.257602 2412 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Bootstrap starting.
I20260811 12:13:56.259835 2413 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Bootstrap starting.
I20260811 12:13:56.261026 2412 tablet_bootstrap.cc:654] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:56.262847 2412 log.cc:824] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:56.263363 2414 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Bootstrap starting.
I20260811 12:13:56.263422 2413 tablet_bootstrap.cc:654] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:56.264755 2413 log.cc:824] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:56.266415 2414 tablet_bootstrap.cc:654] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:56.267656 2413 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: No bootstrap required, opened a new log
I20260811 12:13:56.267637 2414 log.cc:824] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:56.267956 2413 ts_tablet_manager.cc:1397] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.002s
I20260811 12:13:56.268242 2412 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: No bootstrap required, opened a new log
I20260811 12:13:56.268522 2412 ts_tablet_manager.cc:1397] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260811 12:13:56.270975 2414 tablet_bootstrap.cc:492] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: No bootstrap required, opened a new log
I20260811 12:13:56.271281 2414 ts_tablet_manager.cc:1397] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260811 12:13:56.275424 2413 raft_consensus.cc:348] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.275822 2413 raft_consensus.cc:374] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:56.275923 2413 raft_consensus.cc:729] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b1d24fe92e74333843374cebbf19628, State: Initialized, Role: FOLLOWER
I20260811 12:13:56.276487 2413 consensus_queue.cc:260] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.277503 2155 heartbeater.cc:503] Master 127.4.48.254:45507 was elected leader, sending a full tablet report...
I20260811 12:13:56.277966 2413 ts_tablet_manager.cc:1428] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Time spent starting tablet: real 0.010s user 0.004s sys 0.005s
I20260811 12:13:56.279107 2412 raft_consensus.cc:348] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.279541 2412 raft_consensus.cc:374] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:56.279649 2412 raft_consensus.cc:729] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80446c50de5d4948af37656460582e1c, State: Initialized, Role: FOLLOWER
I20260811 12:13:56.280258 2412 consensus_queue.cc:260] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.280611 2414 raft_consensus.cc:348] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.281055 2414 raft_consensus.cc:374] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:56.281219 2414 raft_consensus.cc:729] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 999009e3083348849f4df5a4ba101f73, State: Initialized, Role: FOLLOWER
I20260811 12:13:56.281930 2414 consensus_queue.cc:260] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.282908 2374 heartbeater.cc:503] Master 127.4.48.254:45507 was elected leader, sending a full tablet report...
I20260811 12:13:56.284340 1922 heartbeater.cc:503] Master 127.4.48.254:45507 was elected leader, sending a full tablet report...
I20260811 12:13:56.285004 2412 ts_tablet_manager.cc:1428] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Time spent starting tablet: real 0.016s user 0.011s sys 0.004s
I20260811 12:13:56.285137 2414 ts_tablet_manager.cc:1428] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Time spent starting tablet: real 0.014s user 0.011s sys 0.000s
W20260811 12:13:56.291844 1923 tablet.cc:2369] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:13:56.427090 2376 tablet.cc:2369] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:13:56.490096 2156 tablet.cc:2369] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:13:56.514487 2422 raft_consensus.cc:482] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:13:56.514772 2422 raft_consensus.cc:504] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.516631 2422 leader_election.cc:288] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80446c50de5d4948af37656460582e1c (127.4.48.193:36141), 6b1d24fe92e74333843374cebbf19628 (127.4.48.194:45311)
I20260811 12:13:56.522863 2378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.516860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:36141 (local address 127.4.48.195:42487)
0811 12:13:56.517022 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.517040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.517164 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.517765 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.517771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.517808 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:56.517994 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.518005 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.519226 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.519232 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.520749 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.520762 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.522130 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.522136 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.522308 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.522331 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.522462 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.522589 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":1}
I20260811 12:13:56.522903 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.517150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:42487 (local address 127.4.48.193:36141)
0811 12:13:56.517366 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.517373 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.517393 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:13:56.517468 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.517473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.517562 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.517664 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:56.518159 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.519086 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.520920 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.521992 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.522126 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.522195 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.522510 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.522640 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 12:13:56.523491 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.517360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:57809 (local address 127.4.48.194:45311)
0811 12:13:56.517522 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.517527 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.517932 (+ 405us) server_negotiation.cc:407] Connection header received
0811 12:13:56.518049 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.518054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.518131 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.518204 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:13:56.518805 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.519645 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.521513 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.522454 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.522603 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.522662 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.523146 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.523272 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:13:56.524013 2423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.517232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45311 (local address 127.4.48.195:57809)
0811 12:13:56.517817 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:56.517834 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:56.517946 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:56.518252 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:56.518257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:56.518281 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:13:56.518668 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.518680 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.519820 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.519824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.521327 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:56.521351 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.522780 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.522787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:56.522961 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.522992 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:13:56.523651 (+ 659us) client_negotiation.cc:240] Negotiation successful
0811 12:13:56.523815 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":291,"threads_started":1}
I20260811 12:13:56.523931 1852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556f5aba44534a65a2ff9b4bec90c857" candidate_uuid: "999009e3083348849f4df5a4ba101f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80446c50de5d4948af37656460582e1c" is_pre_election: true
I20260811 12:13:56.524458 1852 raft_consensus.cc:2454] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 999009e3083348849f4df5a4ba101f73 in term 0.
I20260811 12:13:56.524933 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556f5aba44534a65a2ff9b4bec90c857" candidate_uuid: "999009e3083348849f4df5a4ba101f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1d24fe92e74333843374cebbf19628" is_pre_election: true
I20260811 12:13:56.525132 2186 leader_election.cc:302] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80446c50de5d4948af37656460582e1c, 999009e3083348849f4df5a4ba101f73; no voters:
I20260811 12:13:56.525382 2073 raft_consensus.cc:2454] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 999009e3083348849f4df5a4ba101f73 in term 0.
I20260811 12:13:56.525497 2422 raft_consensus.cc:2781] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:13:56.525632 2422 raft_consensus.cc:482] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:13:56.525743 2422 raft_consensus.cc:3037] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:56.527913 2422 raft_consensus.cc:504] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.528445 2422 leader_election.cc:288] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [CANDIDATE]: Term 1 election: Requested vote from peers 80446c50de5d4948af37656460582e1c (127.4.48.193:36141), 6b1d24fe92e74333843374cebbf19628 (127.4.48.194:45311)
I20260811 12:13:56.528899 1852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556f5aba44534a65a2ff9b4bec90c857" candidate_uuid: "999009e3083348849f4df5a4ba101f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80446c50de5d4948af37656460582e1c"
I20260811 12:13:56.528936 2073 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "556f5aba44534a65a2ff9b4bec90c857" candidate_uuid: "999009e3083348849f4df5a4ba101f73" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b1d24fe92e74333843374cebbf19628"
I20260811 12:13:56.529103 2073 raft_consensus.cc:3037] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:56.529095 1852 raft_consensus.cc:3037] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:56.531248 2073 raft_consensus.cc:2454] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 999009e3083348849f4df5a4ba101f73 in term 1.
I20260811 12:13:56.531579 1852 raft_consensus.cc:2454] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 999009e3083348849f4df5a4ba101f73 in term 1.
I20260811 12:13:56.531636 2186 leader_election.cc:302] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b1d24fe92e74333843374cebbf19628, 999009e3083348849f4df5a4ba101f73; no voters:
I20260811 12:13:56.531966 2422 raft_consensus.cc:2781] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:56.532394 2422 raft_consensus.cc:686] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 1 LEADER]: Becoming Leader. State: Replica: 999009e3083348849f4df5a4ba101f73, State: Running, Role: LEADER
I20260811 12:13:56.533106 2422 consensus_queue.cc:237] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } }
I20260811 12:13:56.538010 1615 catalog_manager.cc:5495] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 reported cstate change: term changed from 0 to 1, leader changed from <none> to 999009e3083348849f4df5a4ba101f73 (127.4.48.195). New cstate: current_term: 1 leader_uuid: "999009e3083348849f4df5a4ba101f73" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "999009e3083348849f4df5a4ba101f73" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 40545 } health_report { overall_health: HEALTHY } } }
I20260811 12:13:56.606209 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.601135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35464 (local address 127.4.48.195:40545)
0811 12:13:56.601291 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.601296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.601314 (+ 18us) server_negotiation.cc:407] Connection header received
0811 12:13:56.601516 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.601520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.601607 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.601740 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:13:56.602163 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.602926 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.604320 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.604719 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.604806 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.605233 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:13:56.605711 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:13:56.605822 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.605908 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.605968 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:13:56.614280 1852 raft_consensus.cc:1261] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 1 FOLLOWER]: Refusing update from remote peer 999009e3083348849f4df5a4ba101f73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:56.614281 2073 raft_consensus.cc:1261] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 1 FOLLOWER]: Refusing update from remote peer 999009e3083348849f4df5a4ba101f73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:13:56.615224 2425 consensus_queue.cc:1035] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80446c50de5d4948af37656460582e1c" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 36141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:56.615582 2422 consensus_queue.cc:1035] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b1d24fe92e74333843374cebbf19628" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:13:56.627440 2434 mvcc.cc:205] Tried to move back new op lower bound from 7317300988361981952 to 7317300988043390976. Current Snapshot: MvccSnapshot[applied={T|T < 7317300988361981952}]
I20260811 12:13:56.638695 2435 mvcc.cc:205] Tried to move back new op lower bound from 7317300988361981952 to 7317300988043390976. Current Snapshot: MvccSnapshot[applied={T|T < 7317300988361981952}]
I20260811 12:13:56.639055 1615 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:56.639333 1615 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:13:56.644140 1615 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a51f31b78b3d4b08afb00af44b887bdd: Sending DeleteTablet for 3 replicas of tablet 556f5aba44534a65a2ff9b4bec90c857
I20260811 12:13:56.645366 1826 tablet_service.cc:1514] Processing DeleteTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:56 UTC) from {username='slave'} at 127.0.0.1:49458
I20260811 12:13:56.645452 2046 tablet_service.cc:1514] Processing DeleteTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:56 UTC) from {username='slave'} at 127.0.0.1:42670
I20260811 12:13:56.645563 2263 tablet_service.cc:1514] Processing DeleteTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:56 UTC) from {username='slave'} at 127.0.0.1:35462
I20260811 12:13:56.645861 2045 tablet_service.cc:1514] Processing DeleteTablet for tablet 556f5aba44534a65a2ff9b4bec90c857 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:13:56 UTC) from {username='slave'} at 127.0.0.1:42670
I20260811 12:13:56.646582 2450 tablet_replica.cc:330] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: stopping tablet replica
I20260811 12:13:56.647015 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 1714
I20260811 12:13:56.652112 2450 raft_consensus.cc:2229] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:56.652719 2450 raft_consensus.cc:2258] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:56.654620 2451 tablet_replica.cc:330] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: stopping tablet replica
I20260811 12:13:56.655048 2450 ts_tablet_manager.cc:1905] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:56.655057 2451 raft_consensus.cc:2229] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:13:56.655714 2451 raft_consensus.cc:2258] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:56.657645 2451 ts_tablet_manager.cc:1905] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:56.661231 2449 tablet_replica.cc:330] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: stopping tablet replica
I20260811 12:13:56.661628 2449 raft_consensus.cc:2229] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:13:56.662123 2449 raft_consensus.cc:2258] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:13:56.662269 2450 ts_tablet_manager.cc:1918] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:56.662434 2450 log.cc:1196] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/556f5aba44534a65a2ff9b4bec90c857
I20260811 12:13:56.663033 2450 ts_tablet_manager.cc:1939] T 556f5aba44534a65a2ff9b4bec90c857 P 80446c50de5d4948af37656460582e1c: Deleting consensus metadata
I20260811 12:13:56.664657 2451 ts_tablet_manager.cc:1918] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:56.664824 2451 log.cc:1196] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/556f5aba44534a65a2ff9b4bec90c857
I20260811 12:13:56.665246 1590 catalog_manager.cc:4841] TS 80446c50de5d4948af37656460582e1c (127.4.48.193:36141): tablet 556f5aba44534a65a2ff9b4bec90c857 (table test-table [id=291fd70e58b2462b9409de41043b78a2]) successfully deleted
I20260811 12:13:56.665861 2451 ts_tablet_manager.cc:1939] T 556f5aba44534a65a2ff9b4bec90c857 P 999009e3083348849f4df5a4ba101f73: Deleting consensus metadata
I20260811 12:13:56.666790 2398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.649522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49462 (local address 127.4.48.193:36141)
0811 12:13:56.649694 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.649699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.654309 (+ 4610us) server_negotiation.cc:407] Connection header received
0811 12:13:56.655162 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.655168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.655265 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.655356 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:56.656564 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.657730 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.662564 (+ 4834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.663233 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.663335 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.665109 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:56.665119 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:56.665134 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:56.665836 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:56.665895 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:56.665920 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:56.666018 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.666273 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.666388 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:13:56.667488 1913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:56.667497 1590 catalog_manager.cc:4841] TS 999009e3083348849f4df5a4ba101f73 (127.4.48.195:40545): tablet 556f5aba44534a65a2ff9b4bec90c857 (table test-table [id=291fd70e58b2462b9409de41043b78a2]) successfully deleted
I20260811 12:13:56.668167 2449 ts_tablet_manager.cc:1905] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:13:56.673350 2449 ts_tablet_manager.cc:1918] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:13:56.673601 2449 log.cc:1196] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/556f5aba44534a65a2ff9b4bec90c857
I20260811 12:13:56.674444 2449 ts_tablet_manager.cc:1939] T 556f5aba44534a65a2ff9b4bec90c857 P 6b1d24fe92e74333843374cebbf19628: Deleting consensus metadata
I20260811 12:13:56.676311 1590 catalog_manager.cc:4841] TS 6b1d24fe92e74333843374cebbf19628 (127.4.48.194:45311): tablet 556f5aba44534a65a2ff9b4bec90c857 (table test-table [id=291fd70e58b2462b9409de41043b78a2]) successfully deleted
W20260811 12:13:56.676759 1590 catalog_manager.cc:4808] TS 6b1d24fe92e74333843374cebbf19628 (127.4.48.194:45311): delete failed for tablet 556f5aba44534a65a2ff9b4bec90c857 because the tablet was not found. No further retry: Not found: Tablet not found: 556f5aba44534a65a2ff9b4bec90c857
I20260811 12:13:56.760236 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 1714
I20260811 12:13:56.782850 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 1929
I20260811 12:13:56.790915 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.783808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.4.48.194:45311)
0811 12:13:56.783995 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.784001 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.784021 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:13:56.784152 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.784158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.784248 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.784335 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:56.784736 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.785461 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.787522 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.788352 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.788472 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.789549 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:56.789558 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:56.789575 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:56.790171 (+ 596us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:56.790221 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:56.790241 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:56.790340 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.790474 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.790587 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:13:56.791616 2146 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:56.880084 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 1929
I20260811 12:13:56.908835 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 2159
I20260811 12:13:56.915961 2411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:56.909619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35472 (local address 127.4.48.195:40545)
0811 12:13:56.909812 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:56.909818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:56.909899 (+ 81us) server_negotiation.cc:407] Connection header received
0811 12:13:56.910099 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:56.910105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:56.910190 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:56.910274 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:56.910700 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.911533 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.912821 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:56.913306 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:56.913434 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:56.914542 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:56.914551 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:56.914567 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:13:56.915179 (+ 612us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:56.915224 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:13:56.915245 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:56.915343 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:56.915491 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:13:56.915600 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:13:56.916709 2367 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:57.030941 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 2159
I20260811 12:13:57.056736 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 1569
I20260811 12:13:57.057797 1669 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:13:57.162678 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 1569
2026-08-11T12:13:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2966 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:13:57 dist-test-slave-nj21 krb5kdc[2463](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:13:57 dist-test-slave-nj21 krb5kdc[2463](info): set up 2 sockets
Aug 11 12:13:57 dist-test-slave-nj21 krb5kdc[2463](info): commencing operation
krb5kdc: starting...
W20260811 12:13:58.393026 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:13:58 dist-test-slave-nj21 krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:13:58.659724 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44097
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:44097
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:58.989946 2486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:58.990212 2486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:58.990338 2486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:58.997341 2486 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:13:58.997440 2486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:58.997516 2486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:58.997586 2486 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:13:58.997644 2486 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:13:59.007480 2486 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:44097
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:44097
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:59.008725 2486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:59.009994 2486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:59.021027 2498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:59.021112 2493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:59.021234 2495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:59.022080 2486 server_base.cc:1034] running on GCE node
I20260811 12:13:59.022789 2486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:59.024123 2486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:59.025322 2486 hybrid_clock.cc:648] HybridClock initialized: now 1786450439025271 us; error 36 us; skew 500 ppm
I20260811 12:13:59.025657 2486 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:59 dist-test-slave-nj21 krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:59.028610 2486 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:13:59.029958 2486 webserver.cc:460] Webserver started at http://127.4.48.254:39067/ using document root <none> and password file <none>
I20260811 12:13:59.030587 2486 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:59.030685 2486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:59.030934 2486 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:59.033774 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "5c1b9e26205240b4b76b11143855371f"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "dcce2fe73a65a39835f31308901f31c7"
server_key_iv: "4acccf61ec1b15647c0aa8984057c892"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.034500 2486 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "5c1b9e26205240b4b76b11143855371f"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "dcce2fe73a65a39835f31308901f31c7"
server_key_iv: "4acccf61ec1b15647c0aa8984057c892"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.039456 2486 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:13:59.042748 2509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:59.044023 2486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:13:59.044174 2486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "5c1b9e26205240b4b76b11143855371f"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "dcce2fe73a65a39835f31308901f31c7"
server_key_iv: "4acccf61ec1b15647c0aa8984057c892"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.044348 2486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:59.062095 2486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:59.065855 2486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:59.066103 2486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:59.083047 2486 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:44097
I20260811 12:13:59.083106 2595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:44097 every 8 connection(s)
I20260811 12:13:59.084407 2486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:13:59.087980 2598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:13:59.093482 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 2486
I20260811 12:13:59.093648 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:13:59.094069 4291 external_mini_cluster.cc:1428] Setting key f6e405cd104f89b21fd93922ba351bed
I20260811 12:13:59.098269 2598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: Bootstrap starting.
I20260811 12:13:59.101733 2598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: Neither blocks nor log segments found. Creating new log.
I20260811 12:13:59.102866 2598 log.cc:824] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:13:59.105840 2598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: No bootstrap required, opened a new log
Aug 11 12:13:59 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:59.114794 2598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } }
I20260811 12:13:59.115257 2598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:13:59.115393 2598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1b9e26205240b4b76b11143855371f, State: Initialized, Role: FOLLOWER
I20260811 12:13:59.116143 2598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } }
I20260811 12:13:59.116386 2598 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:13:59.116521 2598 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:13:59.116680 2598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:13:59.117063 2602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:59.095852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.4.48.254:44097)
0811 12:13:59.096633 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:59.096644 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:59.096684 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:13:59.098305 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:59.098327 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:59.098708 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:59.099004 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:59.100806 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.102110 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.103871 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.104583 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.104711 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:59.109636 (+ 4925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:59.109644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:59.109658 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:59.109699 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:59.112771 (+ 3072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:59.113573 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:59.113579 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:59.113585 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:59.113758 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:59.114174 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:59.114177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:59.114179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:59.114697 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:59.114921 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:59.115504 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 12:13:59.115965 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":364,"threads_started":1}
I20260811 12:13:59.119045 2598 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } }
I20260811 12:13:59.119524 2598 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1b9e26205240b4b76b11143855371f; no voters:
I20260811 12:13:59.120205 2598 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:13:59.120308 2604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:13:59.121007 2604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [term 1 LEADER]: Becoming Leader. State: Replica: 5c1b9e26205240b4b76b11143855371f, State: Running, Role: LEADER
I20260811 12:13:59.121518 2604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } }
I20260811 12:13:59.122167 2598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:13:59.124343 2606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1b9e26205240b4b76b11143855371f. Latest consensus state: current_term: 1 leader_uuid: "5c1b9e26205240b4b76b11143855371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } } }
I20260811 12:13:59.124719 2606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:59.125708 2605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c1b9e26205240b4b76b11143855371f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1b9e26205240b4b76b11143855371f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 44097 } } }
I20260811 12:13:59.126088 2605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f [sys.catalog]: This master's current role is: LEADER
I20260811 12:13:59.126451 2610 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:13:59.131330 2610 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:13:59.140482 2610 catalog_manager.cc:1347] Generated new cluster ID: d66046068bd745bfb47ad1ce76badf17
I20260811 12:13:59.140620 2610 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:13:59.159955 2610 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:13:59.161180 2610 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:13:59.169011 2610 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: Generated new TSK 0
I20260811 12:13:59.169827 2610 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:13:59.245102 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:13:59.572448 2637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:13:59.572707 2637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:13:59.572837 2637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:13:59.579629 2637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:13:59.579756 2637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:13:59.579907 2637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:13:59.589062 2637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:13:59.590270 2637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:13:59.591516 2637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:13:59.603608 2649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:59.603657 2655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:13:59.603700 2646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:13:59.604701 2637 server_base.cc:1034] running on GCE node
I20260811 12:13:59.605244 2637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:13:59.606086 2637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:13:59.607291 2637 hybrid_clock.cc:648] HybridClock initialized: now 1786450439607244 us; error 39 us; skew 500 ppm
I20260811 12:13:59.607581 2637 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:13:59 dist-test-slave-nj21 krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:13:59.610522 2637 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:13:59.611845 2637 webserver.cc:460] Webserver started at http://127.4.48.193:38831/ using document root <none> and password file <none>
I20260811 12:13:59.612471 2637 fs_manager.cc:359] Metadata directory not provided
I20260811 12:13:59.612586 2637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:13:59.612869 2637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:13:59.615515 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "f850e91c44da44babb86e999ed324ab2"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "e7970da5ee38520b0d99feee5094e8e4"
server_key_iv: "a80c27a8efae503bbc19ec47476a73e4"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.616429 2637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "f850e91c44da44babb86e999ed324ab2"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "e7970da5ee38520b0d99feee5094e8e4"
server_key_iv: "a80c27a8efae503bbc19ec47476a73e4"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.621196 2637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:13:59.624563 2667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:59.625903 2637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:13:59.626057 2637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "f850e91c44da44babb86e999ed324ab2"
format_stamp: "Formatted at 2026-08-11 12:13:59 on dist-test-slave-nj21"
server_key: "e7970da5ee38520b0d99feee5094e8e4"
server_key_iv: "a80c27a8efae503bbc19ec47476a73e4"
server_key_version: "encryptionkey@0"
I20260811 12:13:59.626214 2637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:13:59.674329 2637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:13:59.678429 2637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:13:59.678731 2637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:13:59.680059 2637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:13:59.681907 2637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:13:59.681995 2637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:59.682147 2637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:13:59.682216 2637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:13:59.712327 2637 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:43245
I20260811 12:13:59.712388 2870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:43245 every 8 connection(s)
I20260811 12:13:59.713536 2637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:13:59.721648 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 2637
I20260811 12:13:59.721830 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:13:59.722239 4291 external_mini_cluster.cc:1428] Setting key cdbd278fc412782127b3d4c47abec2ce
Aug 11 12:13:59 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450439, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:13:59.733261 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:59.715965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:60131 (local address 127.4.48.254:44097)
0811 12:13:59.716414 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0811 12:13:59.716419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:13:59.717328 (+ 909us) server_negotiation.cc:407] Connection header received
0811 12:13:59.719107 (+ 1779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:13:59.719113 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:13:59.719190 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:13:59.719284 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:13:59.720757 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.721571 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.723407 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.723839 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.723971 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:59.727797 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:13:59.727803 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:13:59.727806 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:13:59.727839 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:13:59.730267 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:59.731080 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:59.731084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:59.731086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:59.731204 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:13:59.731642 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:13:59.731645 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:13:59.731647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:13:59.731919 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:13:59.732029 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 12:13:59.732809 (+ 780us) server_negotiation.cc:299] Negotiation successful
0811 12:13:59.732992 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":218,"threads_started":1}
I20260811 12:13:59.734433 2874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:13:59.716503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44097 (local address 127.4.48.193:60131)
0811 12:13:59.717180 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 12:13:59.717217 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 12:13:59.718855 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:13:59.719513 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:13:59.719521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:13:59.720072 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:13:59.720598 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:59.720615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.721727 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.721732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:59.723253 (+ 1521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:13:59.723266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:13:59.724053 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:13:59.724059 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:13:59.724331 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:13:59.725008 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:13:59.725039 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:13:59.727573 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:13:59.730393 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:59.730400 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:59.730414 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:59.730963 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:59.731297 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:13:59.731301 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:13:59.731304 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:13:59.731546 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:13:59.732042 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:13:59.732049 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:13:59.732435 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 12:13:59.732720 (+ 285us) client_negotiation.cc:240] Negotiation successful
0811 12:13:59.733180 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":280,"threads_started":1}
I20260811 12:13:59.735908 2871 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44097
I20260811 12:13:59.736177 2871 heartbeater.cc:464] Registering TS with master...
I20260811 12:13:59.736845 2871 heartbeater.cc:511] Master 127.4.48.254:44097 requested a full tablet report, sending...
I20260811 12:13:59.738842 2539 ts_manager.cc:194] Registered new tserver with Master: f850e91c44da44babb86e999ed324ab2 (127.4.48.193:43245)
I20260811 12:13:59.740332 2539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:60131
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:13:59.783532 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:00.106528 2878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:00.106789 2878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:00.106930 2878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:00.114027 2878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:00.114137 2878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:00.114300 2878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:14:00.124007 2878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:00.125257 2878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:00.126530 2878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:00.139765 2889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:00.139807 2893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:00.139848 2890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:00.140661 2878 server_base.cc:1034] running on GCE node
I20260811 12:14:00.141211 2878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:00.142108 2878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:00.143318 2878 hybrid_clock.cc:648] HybridClock initialized: now 1786450440143268 us; error 38 us; skew 500 ppm
I20260811 12:14:00.143611 2878 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:00 dist-test-slave-nj21 krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:00.146570 2878 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:14:00.148000 2878 webserver.cc:460] Webserver started at http://127.4.48.194:39963/ using document root <none> and password file <none>
I20260811 12:14:00.148662 2878 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:00.148787 2878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:00.149089 2878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:00.151932 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "b4e42e137cdd40baa948897e293724d5"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "3d4f3f85d60be93cdf0de21ebc2ce29b"
server_key_iv: "3f056b48cd02fb426643e5cd5966210c"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.152666 2878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "b4e42e137cdd40baa948897e293724d5"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "3d4f3f85d60be93cdf0de21ebc2ce29b"
server_key_iv: "3f056b48cd02fb426643e5cd5966210c"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.157500 2878 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:14:00.160684 2901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.161899 2878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:14:00.162048 2878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "b4e42e137cdd40baa948897e293724d5"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "3d4f3f85d60be93cdf0de21ebc2ce29b"
server_key_iv: "3f056b48cd02fb426643e5cd5966210c"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.162214 2878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:00.179360 2878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:00.182891 2878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:00.183151 2878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:00.184095 2878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:00.185467 2878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:00.185545 2878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.185667 2878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:00.185730 2878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.215812 2878 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:37257
I20260811 12:14:00.215829 3097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:37257 every 8 connection(s)
I20260811 12:14:00.217024 2878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:14:00.218155 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 2878
I20260811 12:14:00.218338 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:14:00.218766 4291 external_mini_cluster.cc:1428] Setting key 176515affc21c316f527c8349606c8b1
Aug 11 12:14:00 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:00.238152 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.220888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:55133 (local address 127.4.48.254:44097)
0811 12:14:00.221100 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.221105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.222416 (+ 1311us) server_negotiation.cc:407] Connection header received
0811 12:14:00.224341 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.224347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.224425 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.224521 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:00.225934 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.226859 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.228382 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.228956 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.229070 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.232546 (+ 3476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:00.232552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:00.232556 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:00.232592 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:00.235245 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.235981 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.235985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.235987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.236114 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.236515 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.236518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.236520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.236778 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:00.236908 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.237582 (+ 674us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.237826 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:14:00.239079 3101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.221434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44097 (local address 127.4.48.194:55133)
0811 12:14:00.222266 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.222312 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.224091 (+ 1779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.224707 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.224715 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.225218 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:00.225771 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.225788 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.226981 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.226984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.228243 (+ 1259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.228256 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.229150 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.229155 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.229376 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.230014 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:00.230048 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:00.232326 (+ 2278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:00.235378 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:00.235387 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:00.235402 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:00.235863 (+ 461us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:00.236215 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:00.236227 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:00.236231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:00.236403 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:00.236908 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:00.236916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:00.237215 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.237454 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.237935 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":370,"threads_started":1}
I20260811 12:14:00.240617 3098 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44097
I20260811 12:14:00.240952 3098 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:00.241854 3098 heartbeater.cc:511] Master 127.4.48.254:44097 requested a full tablet report, sending...
I20260811 12:14:00.243314 2539 ts_manager.cc:194] Registered new tserver with Master: b4e42e137cdd40baa948897e293724d5 (127.4.48.194:37257)
I20260811 12:14:00.244372 2539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:55133
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:14:00.284261 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:00.606557 3107 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:00.606818 3107 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:00.606950 3107 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:00.613929 3107 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:00.614023 3107 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:00.614171 3107 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:14:00.623900 3107 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:44097
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:00.625150 3107 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:00.626420 3107 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:00.638945 3115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:00.639097 3116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:00.639019 3118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:00.640092 3107 server_base.cc:1034] running on GCE node
I20260811 12:14:00.640640 3107 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:00.641484 3107 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:00.642668 3107 hybrid_clock.cc:648] HybridClock initialized: now 1786450440642603 us; error 59 us; skew 500 ppm
I20260811 12:14:00.642956 3107 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:00 dist-test-slave-nj21 krb5kdc[2463](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:00.645908 3107 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:14:00.647322 3107 webserver.cc:460] Webserver started at http://127.4.48.195:33193/ using document root <none> and password file <none>
I20260811 12:14:00.648057 3107 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:00.648176 3107 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:00.648470 3107 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:00.651264 3107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "75c1e0ae18084fa4bb5fac4fabc61f29"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "504539cce3801b7c6b8bb4628f3f735f"
server_key_iv: "501da05e3f3ba16c7249218db0d25a78"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.652072 3107 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "75c1e0ae18084fa4bb5fac4fabc61f29"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "504539cce3801b7c6b8bb4628f3f735f"
server_key_iv: "501da05e3f3ba16c7249218db0d25a78"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.656942 3107 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 12:14:00.660189 3128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.661383 3107 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 12:14:00.661533 3107 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "75c1e0ae18084fa4bb5fac4fabc61f29"
format_stamp: "Formatted at 2026-08-11 12:14:00 on dist-test-slave-nj21"
server_key: "504539cce3801b7c6b8bb4628f3f735f"
server_key_iv: "501da05e3f3ba16c7249218db0d25a78"
server_key_version: "encryptionkey@0"
I20260811 12:14:00.661696 3107 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:00.682097 3107 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:00.685791 3107 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:00.686034 3107 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:00.686923 3107 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:00.688220 3107 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:00.688301 3107 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.688419 3107 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:00.688472 3107 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:00.718994 3107 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:41489
I20260811 12:14:00.719038 3318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:41489 every 8 connection(s)
I20260811 12:14:00.720191 3107 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:14:00.721205 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 3107
I20260811 12:14:00.721393 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:14:00.721947 4291 external_mini_cluster.cc:1428] Setting key 7a6f13e6c9aa315641a19e48a5155975
Aug 11 12:14:00 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450440, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:00.739362 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.723033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:36651 (local address 127.4.48.254:44097)
0811 12:14:00.723194 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.723200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.724362 (+ 1162us) server_negotiation.cc:407] Connection header received
0811 12:14:00.726228 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.726234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.726347 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.726431 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:00.727875 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.728660 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.730228 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.730619 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.730736 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.734056 (+ 3320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:00.734061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:00.734065 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:00.734098 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:00.736921 (+ 2823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.737571 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.737575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.737578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.737706 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.738033 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.738036 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.738038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.738287 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:00.738396 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.738966 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.739124 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 12:14:00.740504 3325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.723392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:44097 (local address 127.4.48.195:36651)
0811 12:14:00.724208 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.724259 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.725991 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.726665 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.726672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.727154 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:00.727715 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.727738 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.728812 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.728817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.730098 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.730112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.730835 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.730842 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.731071 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.731703 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:00.731744 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:00.733837 (+ 2093us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:00.737043 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:00.737050 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:00.737061 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:00.737471 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:00.737785 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:00.737788 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:00.737790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:00.737952 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:00.738373 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:00.738378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:00.738658 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.738866 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.739226 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":347,"threads_started":1}
I20260811 12:14:00.741708 3319 heartbeater.cc:347] Connected to a master server at 127.4.48.254:44097
I20260811 12:14:00.742005 3319 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:00.742660 3319 heartbeater.cc:511] Master 127.4.48.254:44097 requested a full tablet report, sending...
I20260811 12:14:00.743444 2871 heartbeater.cc:503] Master 127.4.48.254:44097 was elected leader, sending a full tablet report...
I20260811 12:14:00.744180 2539 ts_manager.cc:194] Registered new tserver with Master: 75c1e0ae18084fa4bb5fac4fabc61f29 (127.4.48.195:41489)
I20260811 12:14:00.745072 2539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:36651
I20260811 12:14:00.749401 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:14:00.766049 2873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.753320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48890 (local address 127.4.48.254:44097)
0811 12:14:00.753475 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.753479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.753933 (+ 454us) server_negotiation.cc:407] Connection header received
0811 12:14:00.754085 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.754090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.754147 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.754225 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:00.755638 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.756356 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.757725 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.758122 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.758230 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.760978 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:00.760983 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:00.760986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:00.761016 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:00.763208 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.763997 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.764001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.764004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.764129 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:00.764591 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:00.764594 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:00.764596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:00.764932 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:00.765190 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.765614 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.765785 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:14:00.769562 2539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48890:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:14:00.776288 2539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:14:00.812179 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.790540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.4.48.194:37257)
0811 12:14:00.803433 (+ 12893us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.803441 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.804134 (+ 693us) server_negotiation.cc:407] Connection header received
0811 12:14:00.804246 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.804253 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.804467 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.804657 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:00.805373 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.806710 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.809236 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.810675 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.810816 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811012 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.811714 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.811886 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12744,"thread_start_us":12644,"threads_started":1}
I20260811 12:14:00.812337 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.787940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41489 (local address 127.0.0.1:50840)
0811 12:14:00.799419 (+ 11479us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.799483 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.799701 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.805861 (+ 6160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.805866 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.805892 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:00.806269 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.806282 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.808148 (+ 1866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.808154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.809879 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.809914 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.811500 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.811506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.811666 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811721 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.811808 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.811948 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11192,"mutex_wait_us":3,"thread_start_us":15165,"threads_started":1}
I20260811 12:14:00.812522 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.788151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50840 (local address 127.4.48.195:41489)
0811 12:14:00.805322 (+ 17171us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.805330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.805361 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:14:00.805456 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.805463 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.805670 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.805851 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:00.806505 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.808006 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.810082 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.811291 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.811415 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811639 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.812074 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.812223 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16987,"thread_start_us":16842,"threads_started":1}
I20260811 12:14:00.812781 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.788629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35796 (local address 127.4.48.193:43245)
0811 12:14:00.801824 (+ 13195us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.801834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.803009 (+ 1175us) server_negotiation.cc:407] Connection header received
0811 12:14:00.803615 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.803624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.803931 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.804104 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:00.805192 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.806719 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.809301 (+ 2582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.810675 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.810812 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811044 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.811774 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.811949 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13005,"thread_start_us":13222,"threads_started":1}
I20260811 12:14:00.812934 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.788293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:43245 (local address 127.0.0.1:35796)
0811 12:14:00.802873 (+ 14580us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.802893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.803014 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.804424 (+ 1410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.804431 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.804459 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:00.804955 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.804976 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.806823 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.806829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.809112 (+ 2283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.809129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.811205 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.811212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.811392 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811466 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.811601 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.811802 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14478,"mutex_wait_us":97,"thread_start_us":10695,"threads_started":1}
I20260811 12:14:00.813117 3347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.790042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:37257 (local address 127.0.0.1:37272)
0811 12:14:00.803887 (+ 13845us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.803911 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.804021 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.804680 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.804686 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.804716 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:00.805202 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.805216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.807232 (+ 2016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.807237 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.809084 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.809097 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.811166 (+ 2069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.811173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.811390 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.811466 (+ 76us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.812455 (+ 989us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.812574 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13719,"mutex_wait_us":70,"thread_start_us":13553,"threads_started":1}
I20260811 12:14:00.816519 2755 tablet_service.cc:1467] Processing CreateTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee (DEFAULT_TABLE table=test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:00.816622 3201 tablet_service.cc:1467] Processing CreateTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee (DEFAULT_TABLE table=test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:00.817797 2755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56532f07f94a4aae8e8b8aa1fa3512ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:00.818066 3201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56532f07f94a4aae8e8b8aa1fa3512ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:00.818190 2981 tablet_service.cc:1467] Processing CreateTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee (DEFAULT_TABLE table=test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:00.819540 2981 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 56532f07f94a4aae8e8b8aa1fa3512ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:00.829334 3349 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Bootstrap starting.
I20260811 12:14:00.832270 3350 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Bootstrap starting.
I20260811 12:14:00.833391 3352 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Bootstrap starting.
I20260811 12:14:00.834600 3349 tablet_bootstrap.cc:654] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:00.835717 3350 tablet_bootstrap.cc:654] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:00.835954 3349 log.cc:824] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:00.836728 3350 log.cc:824] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:00.836829 3352 tablet_bootstrap.cc:654] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:00.837972 3352 log.cc:824] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:00.838878 3349 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: No bootstrap required, opened a new log
I20260811 12:14:00.839164 3349 ts_tablet_manager.cc:1397] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 12:14:00.839694 3350 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: No bootstrap required, opened a new log
I20260811 12:14:00.839954 3350 ts_tablet_manager.cc:1397] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:14:00.840482 3352 tablet_bootstrap.cc:492] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: No bootstrap required, opened a new log
I20260811 12:14:00.840711 3352 ts_tablet_manager.cc:1397] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.005s
I20260811 12:14:00.847704 3352 raft_consensus.cc:348] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.848071 3352 raft_consensus.cc:374] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:00.848172 3352 raft_consensus.cc:729] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e42e137cdd40baa948897e293724d5, State: Initialized, Role: FOLLOWER
I20260811 12:14:00.848807 3352 consensus_queue.cc:260] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.849902 3098 heartbeater.cc:503] Master 127.4.48.254:44097 was elected leader, sending a full tablet report...
I20260811 12:14:00.849846 3349 raft_consensus.cc:348] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.850253 3349 raft_consensus.cc:374] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:00.850420 3349 raft_consensus.cc:729] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f850e91c44da44babb86e999ed324ab2, State: Initialized, Role: FOLLOWER
I20260811 12:14:00.850476 3352 ts_tablet_manager.cc:1428] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260811 12:14:00.851133 3349 consensus_queue.cc:260] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.851481 3350 raft_consensus.cc:348] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.851967 3350 raft_consensus.cc:374] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:00.852108 3350 raft_consensus.cc:729] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c1e0ae18084fa4bb5fac4fabc61f29, State: Initialized, Role: FOLLOWER
I20260811 12:14:00.852952 3349 ts_tablet_manager.cc:1428] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260811 12:14:00.852838 3350 consensus_queue.cc:260] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.854672 3319 heartbeater.cc:503] Master 127.4.48.254:44097 was elected leader, sending a full tablet report...
I20260811 12:14:00.855299 3350 ts_tablet_manager.cc:1428] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Time spent starting tablet: real 0.015s user 0.013s sys 0.000s
I20260811 12:14:00.884295 3363 raft_consensus.cc:482] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:14:00.884581 3363 raft_consensus.cc:504] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.886153 3363 leader_election.cc:288] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75c1e0ae18084fa4bb5fac4fabc61f29 (127.4.48.195:41489), f850e91c44da44babb86e999ed324ab2 (127.4.48.193:43245)
I20260811 12:14:00.892778 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.886660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:52755 (local address 127.4.48.195:41489)
0811 12:14:00.886864 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.886870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.887582 (+ 712us) server_negotiation.cc:407] Connection header received
0811 12:14:00.887831 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.887835 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.887894 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.887970 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:00.888543 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.889276 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.891085 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.891910 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.892056 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.892126 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.892408 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.892538 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:14:00.892786 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.886555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:37433 (local address 127.4.48.193:43245)
0811 12:14:00.886775 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.886781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.887650 (+ 869us) server_negotiation.cc:407] Connection header received
0811 12:14:00.887848 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.887853 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.887926 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.888002 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:00.888646 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.889346 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.891091 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.891858 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.892019 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.892095 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.892411 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.892538 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:14:00.892999 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.886613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:43245 (local address 127.4.48.194:37433)
0811 12:14:00.887545 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.887560 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.887708 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.888037 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.888041 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.888065 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:00.888484 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.888494 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.889504 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.889508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.890932 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.890945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.892092 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.892103 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.892256 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.892280 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.892363 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.892538 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":55,"thread_start_us":449,"threads_started":1}
I20260811 12:14:00.893078 3368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.886606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:41489 (local address 127.4.48.194:52755)
0811 12:14:00.887471 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:00.887491 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:00.887630 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:00.887997 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:00.888001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:00.888026 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:00.888403 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.888415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.889396 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.889402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.890937 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:00.890949 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.892119 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.892126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:00.892260 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.892289 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:14:00.892369 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:14:00.892549 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"mutex_wait_us":63,"thread_start_us":551,"threads_started":1}
I20260811 12:14:00.893977 2786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56532f07f94a4aae8e8b8aa1fa3512ee" candidate_uuid: "b4e42e137cdd40baa948897e293724d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f850e91c44da44babb86e999ed324ab2" is_pre_election: true
I20260811 12:14:00.894032 3233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56532f07f94a4aae8e8b8aa1fa3512ee" candidate_uuid: "b4e42e137cdd40baa948897e293724d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" is_pre_election: true
I20260811 12:14:00.894465 2786 raft_consensus.cc:2454] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4e42e137cdd40baa948897e293724d5 in term 0.
I20260811 12:14:00.894479 3233 raft_consensus.cc:2454] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b4e42e137cdd40baa948897e293724d5 in term 0.
I20260811 12:14:00.895066 2912 leader_election.cc:302] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75c1e0ae18084fa4bb5fac4fabc61f29, b4e42e137cdd40baa948897e293724d5; no voters:
I20260811 12:14:00.895421 3363 raft_consensus.cc:2781] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:14:00.895553 3363 raft_consensus.cc:482] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:14:00.895651 3363 raft_consensus.cc:3037] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:00.897801 3363 raft_consensus.cc:504] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.898375 3363 leader_election.cc:288] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [CANDIDATE]: Term 1 election: Requested vote from peers 75c1e0ae18084fa4bb5fac4fabc61f29 (127.4.48.195:41489), f850e91c44da44babb86e999ed324ab2 (127.4.48.193:43245)
I20260811 12:14:00.898803 3233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56532f07f94a4aae8e8b8aa1fa3512ee" candidate_uuid: "b4e42e137cdd40baa948897e293724d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29"
I20260811 12:14:00.898830 2786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "56532f07f94a4aae8e8b8aa1fa3512ee" candidate_uuid: "b4e42e137cdd40baa948897e293724d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f850e91c44da44babb86e999ed324ab2"
I20260811 12:14:00.899015 2786 raft_consensus.cc:3037] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:00.899015 3233 raft_consensus.cc:3037] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:00.902146 2786 raft_consensus.cc:2454] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4e42e137cdd40baa948897e293724d5 in term 1.
I20260811 12:14:00.902146 3233 raft_consensus.cc:2454] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4e42e137cdd40baa948897e293724d5 in term 1.
I20260811 12:14:00.902539 2904 leader_election.cc:302] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4e42e137cdd40baa948897e293724d5, f850e91c44da44babb86e999ed324ab2; no voters:
I20260811 12:14:00.902812 3363 raft_consensus.cc:2781] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:00.903216 3363 raft_consensus.cc:686] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 1 LEADER]: Becoming Leader. State: Replica: b4e42e137cdd40baa948897e293724d5, State: Running, Role: LEADER
I20260811 12:14:00.903645 3363 consensus_queue.cc:237] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } }
I20260811 12:14:00.908217 2537 catalog_manager.cc:5495] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b4e42e137cdd40baa948897e293724d5 (127.4.48.194). New cstate: current_term: 1 leader_uuid: "b4e42e137cdd40baa948897e293724d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b4e42e137cdd40baa948897e293724d5" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 37257 } health_report { overall_health: HEALTHY } } }
I20260811 12:14:00.937673 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.932589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.4.48.194:37257)
0811 12:14:00.932757 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.932773 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.932962 (+ 189us) server_negotiation.cc:407] Connection header received
0811 12:14:00.933119 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.933122 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.933199 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.933314 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:00.933704 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.934402 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.935753 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:00.936180 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:00.936280 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:00.936720 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:00.937214 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:00.937338 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:00.937417 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 12:14:00.937457 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:14:00.945775 3233 raft_consensus.cc:1261] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 1 FOLLOWER]: Refusing update from remote peer b4e42e137cdd40baa948897e293724d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:00.945773 2786 raft_consensus.cc:1261] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 1 FOLLOWER]: Refusing update from remote peer b4e42e137cdd40baa948897e293724d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:00.946775 3372 consensus_queue.cc:1035] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f850e91c44da44babb86e999ed324ab2" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 43245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:00.947111 3363 consensus_queue.cc:1035] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75c1e0ae18084fa4bb5fac4fabc61f29" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 41489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:00.959760 3381 mvcc.cc:205] Tried to move back new op lower bound from 7317301006104141824 to 7317301005945233408. Current Snapshot: MvccSnapshot[applied={T|T < 7317301006104141824}]
I20260811 12:14:00.962184 3379 mvcc.cc:205] Tried to move back new op lower bound from 7317301006104141824 to 7317301005945233408. Current Snapshot: MvccSnapshot[applied={T|T < 7317301006104141824}]
I20260811 12:14:00.974821 2537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:00.975070 2537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48890:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:00.978390 2537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5c1b9e26205240b4b76b11143855371f: Sending DeleteTablet for 3 replicas of tablet 56532f07f94a4aae8e8b8aa1fa3512ee
I20260811 12:14:00.979472 3201 tablet_service.cc:1514] Processing DeleteTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:00 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:50840
I20260811 12:14:00.979477 2755 tablet_service.cc:1514] Processing DeleteTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:00 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:35796
I20260811 12:14:00.980202 2981 tablet_service.cc:1514] Processing DeleteTablet for tablet 56532f07f94a4aae8e8b8aa1fa3512ee with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:00 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:37272
I20260811 12:14:00.980857 3399 tablet_replica.cc:330] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: stopping tablet replica
I20260811 12:14:00.981310 3399 raft_consensus.cc:2229] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:00.981854 3399 raft_consensus.cc:2258] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:00.983553 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 2637
I20260811 12:14:00.984047 3399 ts_tablet_manager.cc:1905] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:00.990999 3399 ts_tablet_manager.cc:1918] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:00.991166 3399 log.cc:1196] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/56532f07f94a4aae8e8b8aa1fa3512ee
I20260811 12:14:00.991714 3399 ts_tablet_manager.cc:1939] T 56532f07f94a4aae8e8b8aa1fa3512ee P f850e91c44da44babb86e999ed324ab2: Deleting consensus metadata
I20260811 12:14:00.992648 3404 tablet_replica.cc:330] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: stopping tablet replica
I20260811 12:14:00.993105 3404 raft_consensus.cc:2229] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:14:00.993623 2510 catalog_manager.cc:4841] TS f850e91c44da44babb86e999ed324ab2 (127.4.48.193:43245): tablet 56532f07f94a4aae8e8b8aa1fa3512ee (table test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]) successfully deleted
I20260811 12:14:00.993811 3400 tablet_replica.cc:330] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: stopping tablet replica
I20260811 12:14:00.994261 3400 raft_consensus.cc:2229] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:00.994534 3404 raft_consensus.cc:2258] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:00.994778 3400 raft_consensus.cc:2258] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:00.996858 3400 ts_tablet_manager.cc:1905] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:00.996997 3404 ts_tablet_manager.cc:1905] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:01.003587 3400 ts_tablet_manager.cc:1918] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:01.003854 3400 log.cc:1196] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/56532f07f94a4aae8e8b8aa1fa3512ee
I20260811 12:14:01.004217 3404 ts_tablet_manager.cc:1918] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:01.004411 3400 ts_tablet_manager.cc:1939] T 56532f07f94a4aae8e8b8aa1fa3512ee P 75c1e0ae18084fa4bb5fac4fabc61f29: Deleting consensus metadata
I20260811 12:14:01.004436 3404 log.cc:1196] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/56532f07f94a4aae8e8b8aa1fa3512ee
I20260811 12:14:01.004979 3404 ts_tablet_manager.cc:1939] T 56532f07f94a4aae8e8b8aa1fa3512ee P b4e42e137cdd40baa948897e293724d5: Deleting consensus metadata
I20260811 12:14:01.006273 2516 catalog_manager.cc:4841] TS 75c1e0ae18084fa4bb5fac4fabc61f29 (127.4.48.195:41489): tablet 56532f07f94a4aae8e8b8aa1fa3512ee (table test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]) successfully deleted
I20260811 12:14:01.006654 2511 catalog_manager.cc:4841] TS b4e42e137cdd40baa948897e293724d5 (127.4.48.194:37257): tablet 56532f07f94a4aae8e8b8aa1fa3512ee (table test-table [id=1178fc5fd31b4759bff61f1c1ba0bcc2]) successfully deleted
Aug 11 12:14:01 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:14:01.016563 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:00.986360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35808 (local address 127.4.48.193:43245)
0811 12:14:00.986526 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:00.986532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:00.999252 (+ 12720us) server_negotiation.cc:407] Connection header received
0811 12:14:00.999759 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:00.999765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:00.999857 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:00.999930 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:01.002208 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.003300 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.005438 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.006040 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.006139 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:01.010942 (+ 4803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:01.010955 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:01.010971 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:01.011015 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:01.013774 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.014595 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.014601 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.014603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.014732 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.015105 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.015107 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.015109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.015465 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:01.015717 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:01.016090 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 12:14:01.016250 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:14:01.017252 2863 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:01.113669 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 2637
I20260811 12:14:01.153296 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 2878
Aug 11 12:14:01 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:14:01.170084 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:01.154041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37298 (local address 127.4.48.194:37257)
0811 12:14:01.154218 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:01.154224 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:01.154297 (+ 73us) server_negotiation.cc:407] Connection header received
0811 12:14:01.154485 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:01.154489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:01.154560 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:01.154666 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:01.155995 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.156841 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.158212 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.158654 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.158766 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:01.163345 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:01.163359 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:01.163379 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:01.163432 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:01.167243 (+ 3811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.167933 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.167943 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.167946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.168121 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.168447 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.168452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.168456 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.168925 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:01.169267 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:01.169460 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 12:14:01.169681 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:14:01.170855 3085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:01.282260 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 2878
I20260811 12:14:01.307952 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 3107
Aug 11 12:14:01 dist-test-slave-nj21 krb5kdc[2463](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:14:01.321906 3342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:01.308658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.4.48.195:41489)
0811 12:14:01.308811 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:01.308815 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:01.308837 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:14:01.308991 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:01.308995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:01.309071 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:01.309155 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:01.310425 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.311179 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.312362 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:01.312744 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:01.312833 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:01.317204 (+ 4371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:01.317220 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:01.317238 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:01.317270 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:01.319712 (+ 2442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.320286 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.320293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.320296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.320429 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:01.320689 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:01.320692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:01.320694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:01.321042 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:01.321265 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:01.321480 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 12:14:01.321646 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:14:01.322540 3313 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:01.408806 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 3107
I20260811 12:14:01.431195 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 2486
I20260811 12:14:01.432309 2588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:01.540916 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 2486
2026-08-11T12:14:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4379 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T12:14:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:01Z Disabled control of system clock
I20260811 12:14:01.627882 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:35587
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:42275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:35587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:14:01.958276 3417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:01.958549 3417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:01.958637 3417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:01.966090 3417 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:01.966217 3417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:01.966272 3417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:01.966338 3417 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:01.966390 3417 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:01.975687 3417 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:35587
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:35587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:01.976996 3417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:01.978972 3417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:01.990329 3427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:01.990374 3428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:01.990412 3430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:01.991338 3417 server_base.cc:1034] running on GCE node
I20260811 12:14:01.992133 3417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:01.993444 3417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:01.994645 3417 hybrid_clock.cc:648] HybridClock initialized: now 1786450441994594 us; error 44 us; skew 500 ppm
I20260811 12:14:01.994972 3417 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:01.997087 3417 webserver.cc:460] Webserver started at http://127.4.48.254:43349/ using document root <none> and password file <none>
I20260811 12:14:01.997721 3417 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:01.997838 3417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:01.998140 3417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:02.001022 3417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "397737968d664df0b49b68f6dad8863d"
format_stamp: "Formatted at 2026-08-11 12:14:01 on dist-test-slave-nj21"
server_key: "c97656339b78b3abcacda3604fa68135"
server_key_iv: "749c12ec091da981cc324532c4dfc1e4"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.001808 3417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "397737968d664df0b49b68f6dad8863d"
format_stamp: "Formatted at 2026-08-11 12:14:01 on dist-test-slave-nj21"
server_key: "c97656339b78b3abcacda3604fa68135"
server_key_iv: "749c12ec091da981cc324532c4dfc1e4"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.006522 3417 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 12:14:02.009720 3440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.010969 3417 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:14:02.011130 3417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "397737968d664df0b49b68f6dad8863d"
format_stamp: "Formatted at 2026-08-11 12:14:01 on dist-test-slave-nj21"
server_key: "c97656339b78b3abcacda3604fa68135"
server_key_iv: "749c12ec091da981cc324532c4dfc1e4"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.011322 3417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:02.026136 3417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:02.026893 3417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:02.027160 3417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:02.044414 3417 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:35587
I20260811 12:14:02.044458 3540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:35587 every 8 connection(s)
I20260811 12:14:02.045647 3417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:02.049145 3541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:02.051818 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 3417
I20260811 12:14:02.051990 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:02.052495 4291 external_mini_cluster.cc:1428] Setting key e35c7c19b1529981e0e7894a658cab1f
I20260811 12:14:02.060276 3541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: Bootstrap starting.
I20260811 12:14:02.063114 3541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:02.064193 3541 log.cc:824] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:02.065243 3546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:02.054448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55330 (local address 127.4.48.254:35587)
0811 12:14:02.055400 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:02.055410 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:02.055451 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:02.057083 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:02.057105 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:02.057539 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:02.057901 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:02.058438 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.059655 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.061000 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.061675 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.061852 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:02.062894 (+ 1042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:02.062902 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:02.062915 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:02.063431 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:02.063531 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:02.063553 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:02.063650 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:02.063888 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 12:14:02.064159 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":364,"threads_started":1}
I20260811 12:14:02.066872 3541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: No bootstrap required, opened a new log
I20260811 12:14:02.073717 3541 raft_consensus.cc:348] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } }
I20260811 12:14:02.074049 3541 raft_consensus.cc:374] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:02.074139 3541 raft_consensus.cc:729] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 397737968d664df0b49b68f6dad8863d, State: Initialized, Role: FOLLOWER
I20260811 12:14:02.074672 3541 consensus_queue.cc:260] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } }
I20260811 12:14:02.074894 3541 raft_consensus.cc:388] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:02.075014 3541 raft_consensus.cc:482] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:02.075165 3541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:02.077245 3541 raft_consensus.cc:504] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } }
I20260811 12:14:02.077692 3541 leader_election.cc:302] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 397737968d664df0b49b68f6dad8863d; no voters:
I20260811 12:14:02.078218 3541 leader_election.cc:288] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:02.078346 3548 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:02.079025 3548 raft_consensus.cc:686] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [term 1 LEADER]: Becoming Leader. State: Replica: 397737968d664df0b49b68f6dad8863d, State: Running, Role: LEADER
I20260811 12:14:02.079487 3548 consensus_queue.cc:237] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } }
I20260811 12:14:02.080046 3541 sys_catalog.cc:564] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:02.082077 3549 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "397737968d664df0b49b68f6dad8863d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } } }
I20260811 12:14:02.082197 3551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 397737968d664df0b49b68f6dad8863d. Latest consensus state: current_term: 1 leader_uuid: "397737968d664df0b49b68f6dad8863d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "397737968d664df0b49b68f6dad8863d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35587 } } }
I20260811 12:14:02.082705 3551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:02.083613 3549 sys_catalog.cc:458] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:02.087004 3569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:02.091037 3569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:02.099063 3569 catalog_manager.cc:1347] Generated new cluster ID: dae1e3a9614e4f23bb5f08452235d6a5
I20260811 12:14:02.099213 3569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:02.113690 3569 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:02.115010 3569 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:02.125699 3569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: Generated new TSK 0
I20260811 12:14:02.126503 3569 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:02.136756 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--builtin_ntp_servers=127.4.48.212:42275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:14:02.474535 3582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:02.474790 3582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:02.474924 3582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:02.481998 3582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:02.482087 3582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:02.482234 3582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:14:02.490839 3582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:02.492062 3582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:02.493773 3582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:02.505978 3590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:02.506013 3591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:02.505941 3593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:02.506973 3582 server_base.cc:1034] running on GCE node
I20260811 12:14:02.507557 3582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:02.508452 3582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:02.509647 3582 hybrid_clock.cc:648] HybridClock initialized: now 1786450442509599 us; error 43 us; skew 500 ppm
I20260811 12:14:02.509932 3582 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:02.512054 3582 webserver.cc:460] Webserver started at http://127.4.48.193:37197/ using document root <none> and password file <none>
I20260811 12:14:02.512688 3582 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:02.512786 3582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:02.513082 3582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:02.515842 3582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "7eff668bbbef458b9fe90a2024be7a47"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "3904a786a6cdad312957be3b05c09622"
server_key_iv: "95b4691a6c5cdd8289b6be72f0bf2f4c"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.516611 3582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "7eff668bbbef458b9fe90a2024be7a47"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "3904a786a6cdad312957be3b05c09622"
server_key_iv: "95b4691a6c5cdd8289b6be72f0bf2f4c"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.521237 3582 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 12:14:02.524451 3601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.525617 3582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:14:02.525768 3582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "7eff668bbbef458b9fe90a2024be7a47"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "3904a786a6cdad312957be3b05c09622"
server_key_iv: "95b4691a6c5cdd8289b6be72f0bf2f4c"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.525960 3582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:02.546627 3582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:02.547336 3582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:02.547608 3582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:02.548528 3582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:02.549942 3582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:02.550032 3582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.550135 3582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:02.550212 3582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.580350 3582 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:45871
I20260811 12:14:02.580428 3815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:45871 every 8 connection(s)
I20260811 12:14:02.581676 3582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:14:02.589964 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 3582
I20260811 12:14:02.590106 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:14:02.590519 4291 external_mini_cluster.cc:1428] Setting key 132e8dac8ce7871b037d94112feabc08
I20260811 12:14:02.593222 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--builtin_ntp_servers=127.4.48.212:42275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:14:02.593987 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:02.584385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:51769 (local address 127.4.48.254:35587)
0811 12:14:02.584787 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:02.584793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:02.585831 (+ 1038us) server_negotiation.cc:407] Connection header received
0811 12:14:02.587696 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:02.587701 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:02.587785 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:02.587891 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:02.588925 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.589636 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.591221 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.591704 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.591870 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:02.593135 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:02.593140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:02.593145 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:02.593162 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:02.593202 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:02.593213 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:02.593289 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:02.593538 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 12:14:02.593635 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":293,"thread_start_us":202,"threads_started":1}
I20260811 12:14:02.594861 3821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:02.584804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35587 (local address 127.4.48.193:51769)
0811 12:14:02.585678 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:02.585720 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:02.587415 (+ 1695us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:02.588114 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:02.588122 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:02.588212 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:02.588756 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:02.588773 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.589789 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.589793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:02.591080 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:02.591093 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:02.591966 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:02.591975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:02.592206 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:02.592847 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:02.592849 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:02.592887 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:02.592964 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:02.593300 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:02.593312 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:02.593321 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:14:02.593510 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 12:14:02.593873 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":242,"threads_started":1}
I20260811 12:14:02.596271 3818 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35587
I20260811 12:14:02.596583 3818 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:02.597409 3818 heartbeater.cc:511] Master 127.4.48.254:35587 requested a full tablet report, sending...
I20260811 12:14:02.599475 3474 ts_manager.cc:194] Registered new tserver with Master: 7eff668bbbef458b9fe90a2024be7a47 (127.4.48.193:45871)
I20260811 12:14:02.600986 3474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.193:51769
W20260811 12:14:02.924146 3822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:02.924420 3822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:02.924546 3822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:02.931764 3822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:02.931865 3822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:02.932005 3822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:14:02.940726 3822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:02.941938 3822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:02.943732 3822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:02.955928 3837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:02.955977 3828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:02.956002 3835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:02.956877 3822 server_base.cc:1034] running on GCE node
I20260811 12:14:02.957432 3822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:02.958299 3822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:02.959502 3822 hybrid_clock.cc:648] HybridClock initialized: now 1786450442959462 us; error 25 us; skew 500 ppm
I20260811 12:14:02.959827 3822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:02.962073 3822 webserver.cc:460] Webserver started at http://127.4.48.194:40899/ using document root <none> and password file <none>
I20260811 12:14:02.962707 3822 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:02.962805 3822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:02.963119 3822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:02.965831 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "901bca4f8b7448e08b430f978c50898a"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "0384071a6b1d68b673ab3dc111353c04"
server_key_iv: "a78e5a9b1ca08dc4c44772f06511ae81"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.966558 3822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "901bca4f8b7448e08b430f978c50898a"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "0384071a6b1d68b673ab3dc111353c04"
server_key_iv: "a78e5a9b1ca08dc4c44772f06511ae81"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.971225 3822 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 12:14:02.974409 3849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.975615 3822 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:14:02.975803 3822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "901bca4f8b7448e08b430f978c50898a"
format_stamp: "Formatted at 2026-08-11 12:14:02 on dist-test-slave-nj21"
server_key: "0384071a6b1d68b673ab3dc111353c04"
server_key_iv: "a78e5a9b1ca08dc4c44772f06511ae81"
server_key_version: "encryptionkey@0"
I20260811 12:14:02.975991 3822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:02.996519 3822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:02.997215 3822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:02.997480 3822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:02.998349 3822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:02.999714 3822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:02.999809 3822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:02.999909 3822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:02.999979 3822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:03.029304 3822 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:45963
I20260811 12:14:03.029377 4052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:45963 every 8 connection(s)
I20260811 12:14:03.030524 3822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:14:03.036036 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 3822
I20260811 12:14:03.036213 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:14:03.036625 4291 external_mini_cluster.cc:1428] Setting key 29ae2d304137429c598117eb3b1f162e
I20260811 12:14:03.039706 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--builtin_ntp_servers=127.4.48.212:42275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:14:03.043097 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.032932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:37359 (local address 127.4.48.254:35587)
0811 12:14:03.033101 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.033105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.034356 (+ 1251us) server_negotiation.cc:407] Connection header received
0811 12:14:03.036176 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.036182 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.036270 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.036367 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:03.037410 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.038428 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.040565 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.040952 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.041045 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.042447 (+ 1402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:03.042451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:03.042454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:03.042470 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:03.042501 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:03.042512 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:03.042571 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.042788 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.042872 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:14:03.044286 4057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.033367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35587 (local address 127.4.48.194:37359)
0811 12:14:03.034207 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.034249 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.035921 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.036587 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.036595 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.036674 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:03.037227 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.037246 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.038586 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.038591 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.040431 (+ 1840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.040446 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.041167 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.041174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.041450 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.042116 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:03.042120 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:03.042182 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:03.042275 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:03.042588 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:03.042597 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:03.042605 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.042775 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.043113 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":261,"threads_started":1}
I20260811 12:14:03.045734 4055 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35587
I20260811 12:14:03.046032 4055 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:03.046868 4055 heartbeater.cc:511] Master 127.4.48.254:35587 requested a full tablet report, sending...
I20260811 12:14:03.048440 3474 ts_manager.cc:194] Registered new tserver with Master: 901bca4f8b7448e08b430f978c50898a (127.4.48.194:45963)
I20260811 12:14:03.049562 3474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.194:37359
W20260811 12:14:03.370612 4058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:03.370865 4058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:03.370980 4058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:03.377892 4058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:03.377982 4058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:03.378127 4058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:14:03.386992 4058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35587
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:03.388221 4058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:03.389947 4058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:03.402490 4072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:03.402412 4077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:03.402458 4073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:03.403262 4058 server_base.cc:1034] running on GCE node
I20260811 12:14:03.403930 4058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:03.404798 4058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:03.405993 4058 hybrid_clock.cc:648] HybridClock initialized: now 1786450443405952 us; error 39 us; skew 500 ppm
I20260811 12:14:03.406275 4058 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:03.408355 4058 webserver.cc:460] Webserver started at http://127.4.48.195:42033/ using document root <none> and password file <none>
I20260811 12:14:03.408982 4058 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:03.409082 4058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:03.409399 4058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:03.412171 4058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "1ebd31ecaea34273b8aca12e98c84c0e"
format_stamp: "Formatted at 2026-08-11 12:14:03 on dist-test-slave-nj21"
server_key: "a27cafb983ac250a571a925344439ff4"
server_key_iv: "e9034896258b60fa663020cf49223ae5"
server_key_version: "encryptionkey@0"
I20260811 12:14:03.412909 4058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "1ebd31ecaea34273b8aca12e98c84c0e"
format_stamp: "Formatted at 2026-08-11 12:14:03 on dist-test-slave-nj21"
server_key: "a27cafb983ac250a571a925344439ff4"
server_key_iv: "e9034896258b60fa663020cf49223ae5"
server_key_version: "encryptionkey@0"
I20260811 12:14:03.417644 4058 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 12:14:03.420873 4085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:03.422012 4058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:14:03.422152 4058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "1ebd31ecaea34273b8aca12e98c84c0e"
format_stamp: "Formatted at 2026-08-11 12:14:03 on dist-test-slave-nj21"
server_key: "a27cafb983ac250a571a925344439ff4"
server_key_iv: "e9034896258b60fa663020cf49223ae5"
server_key_version: "encryptionkey@0"
I20260811 12:14:03.422336 4058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:03.442549 4058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:03.443269 4058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:03.443548 4058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:03.444465 4058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:03.445796 4058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:03.445890 4058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:03.445992 4058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:03.446064 4058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:03.475203 4058 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:39185
I20260811 12:14:03.475265 4273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:39185 every 8 connection(s)
I20260811 12:14:03.476397 4058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:14:03.481431 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4058
I20260811 12:14:03.481573 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:14:03.482023 4291 external_mini_cluster.cc:1428] Setting key 88568593a9860f207d30b8796e69b5de
I20260811 12:14:03.488364 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.478796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:39447 (local address 127.4.48.254:35587)
0811 12:14:03.478999 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.479005 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.480375 (+ 1370us) server_negotiation.cc:407] Connection header received
0811 12:14:03.482194 (+ 1819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.482200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.482286 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.482402 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:03.483569 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.484453 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.486083 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.486492 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.486594 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.487719 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:03.487726 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:03.487732 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:03.487754 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:03.487789 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:03.487802 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:03.487869 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.488033 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.488109 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:14:03.489248 4280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.479296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35587 (local address 127.4.48.195:39447)
0811 12:14:03.480227 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.480271 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.481923 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.482602 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.482612 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.482727 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:03.483412 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.483431 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.484602 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.484608 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.485965 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.485976 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.486665 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.486669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.486848 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.487420 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:03.487423 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:03.487458 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:03.487529 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:03.487876 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:03.487883 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:03.487889 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.488019 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.488281 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":300,"threads_started":1}
I20260811 12:14:03.490506 4274 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35587
I20260811 12:14:03.490751 4274 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:03.491410 4274 heartbeater.cc:511] Master 127.4.48.254:35587 requested a full tablet report, sending...
I20260811 12:14:03.492748 3474 ts_manager.cc:194] Registered new tserver with Master: 1ebd31ecaea34273b8aca12e98c84c0e (127.4.48.195:39185)
I20260811 12:14:03.493513 3474 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.195:39447
I20260811 12:14:03.496865 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:14:03.505932 3820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.500747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55342 (local address 127.4.48.254:35587)
0811 12:14:03.500888 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.500892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.501381 (+ 489us) server_negotiation.cc:407] Connection header received
0811 12:14:03.501516 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.501520 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.501586 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.501656 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:03.502074 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.502701 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.503998 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.504380 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.504469 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.505397 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:03.505402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:03.505405 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:03.505421 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:03.505450 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:03.505461 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:03.505567 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.505675 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.505716 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:14:03.509285 3474 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55342:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:14:03.515928 3474 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:14:03.537482 4307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.528122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:45963 (local address 127.0.0.1:42644)
0811 12:14:03.528822 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.528839 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.528963 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.530125 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.530131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.530156 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:03.530578 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.530591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.532900 (+ 2309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.532907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.535142 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.535158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.536548 (+ 1390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.536555 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.536737 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.536792 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.536926 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.537094 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":1,"thread_start_us":430,"threads_started":1}
I20260811 12:14:03.537783 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.527014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.4.48.193:45871)
0811 12:14:03.528303 (+ 1289us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.528311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.528351 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:14:03.528454 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.528461 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.528703 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.528871 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:03.529949 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.531467 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.535090 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.536482 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.536614 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.536841 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.537277 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.537459 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1122,"thread_start_us":1008,"threads_started":1}
I20260811 12:14:03.537546 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.528472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.4.48.194:45963)
0811 12:14:03.529524 (+ 1052us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.529532 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.529561 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:14:03.529659 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.529666 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.529905 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.530848 (+ 943us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:03.531137 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.532715 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.535367 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.536374 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.536493 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.536763 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.537002 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.537179 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":862,"thread_start_us":707,"threads_started":1}
I20260811 12:14:03.538749 4305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.526847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:45871 (local address 127.0.0.1:43156)
0811 12:14:03.527512 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.527551 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.527748 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.529214 (+ 1466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.529221 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.529250 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:03.529732 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.529747 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.531638 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.531644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.534837 (+ 3193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.534852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.536899 (+ 2047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.536906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.537075 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.537105 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.537774 (+ 669us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.538416 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":710,"thread_start_us":261,"threads_started":1}
I20260811 12:14:03.541590 4309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.529481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39185 (local address 127.0.0.1:41218)
0811 12:14:03.533991 (+ 4510us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.534012 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.534120 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.535018 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.535022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.535052 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:03.535399 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.535411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.537242 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.537248 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.539170 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.539185 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.540892 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.540898 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.541068 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.541101 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.541195 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.541309 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4393,"thread_start_us":4234,"threads_started":1}
I20260811 12:14:03.541764 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.529868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41218 (local address 127.4.48.195:39185)
0811 12:14:03.534252 (+ 4384us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.534260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.534291 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:14:03.534371 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.534390 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.534580 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.534756 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:03.535587 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.537063 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.539365 (+ 2302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.540688 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.540823 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.541063 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.541270 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.541430 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4225,"thread_start_us":4877,"threads_started":1}
I20260811 12:14:03.542045 3938 tablet_service.cc:1467] Processing CreateTablet for tablet ee3118598e614437854f0dbf8ec4cb17 (DEFAULT_TABLE table=test-table [id=0aa7f7ab506342658c658f09b1d76bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:03.542040 3700 tablet_service.cc:1467] Processing CreateTablet for tablet ee3118598e614437854f0dbf8ec4cb17 (DEFAULT_TABLE table=test-table [id=0aa7f7ab506342658c658f09b1d76bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:03.543274 3700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3118598e614437854f0dbf8ec4cb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:03.543289 3938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3118598e614437854f0dbf8ec4cb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:03.546555 4164 tablet_service.cc:1467] Processing CreateTablet for tablet ee3118598e614437854f0dbf8ec4cb17 (DEFAULT_TABLE table=test-table [id=0aa7f7ab506342658c658f09b1d76bfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:03.547828 4164 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee3118598e614437854f0dbf8ec4cb17. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:03.554576 4315 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Bootstrap starting.
I20260811 12:14:03.558025 4315 tablet_bootstrap.cc:654] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:03.559083 4315 log.cc:824] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:03.559437 4319 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Bootstrap starting.
I20260811 12:14:03.562650 4315 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: No bootstrap required, opened a new log
I20260811 12:14:03.562707 4319 tablet_bootstrap.cc:654] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:03.562942 4315 ts_tablet_manager.cc:1397] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 12:14:03.563375 4322 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Bootstrap starting.
I20260811 12:14:03.563977 4319 log.cc:824] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:03.566359 4322 tablet_bootstrap.cc:654] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:03.567227 4322 log.cc:824] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:03.567579 4319 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: No bootstrap required, opened a new log
I20260811 12:14:03.567934 4319 ts_tablet_manager.cc:1397] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:14:03.569571 4322 tablet_bootstrap.cc:492] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: No bootstrap required, opened a new log
I20260811 12:14:03.569788 4322 ts_tablet_manager.cc:1397] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 12:14:03.573063 4315 raft_consensus.cc:348] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.573503 4315 raft_consensus.cc:374] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:03.573603 4315 raft_consensus.cc:729] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7eff668bbbef458b9fe90a2024be7a47, State: Initialized, Role: FOLLOWER
I20260811 12:14:03.574214 4315 consensus_queue.cc:260] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.575336 3818 heartbeater.cc:503] Master 127.4.48.254:35587 was elected leader, sending a full tablet report...
I20260811 12:14:03.576026 4315 ts_tablet_manager.cc:1428] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260811 12:14:03.577199 4322 raft_consensus.cc:348] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.577519 4322 raft_consensus.cc:374] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:03.577615 4322 raft_consensus.cc:729] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ebd31ecaea34273b8aca12e98c84c0e, State: Initialized, Role: FOLLOWER
I20260811 12:14:03.578125 4322 consensus_queue.cc:260] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.578575 4319 raft_consensus.cc:348] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.579031 4319 raft_consensus.cc:374] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:03.579164 4319 raft_consensus.cc:729] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 901bca4f8b7448e08b430f978c50898a, State: Initialized, Role: FOLLOWER
I20260811 12:14:03.579420 4274 heartbeater.cc:503] Master 127.4.48.254:35587 was elected leader, sending a full tablet report...
I20260811 12:14:03.579759 4322 ts_tablet_manager.cc:1428] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Time spent starting tablet: real 0.010s user 0.005s sys 0.005s
I20260811 12:14:03.580036 4319 consensus_queue.cc:260] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
W20260811 12:14:03.583685 3819 tablet.cc:2369] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:03.583757 4055 heartbeater.cc:503] Master 127.4.48.254:35587 was elected leader, sending a full tablet report...
I20260811 12:14:03.585325 4319 ts_tablet_manager.cc:1428] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Time spent starting tablet: real 0.017s user 0.003s sys 0.011s
I20260811 12:14:03.709224 4335 raft_consensus.cc:482] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:14:03.709563 4335 raft_consensus.cc:504] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.711252 4335 leader_election.cc:288] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7eff668bbbef458b9fe90a2024be7a47 (127.4.48.193:45871), 1ebd31ecaea34273b8aca12e98c84c0e (127.4.48.195:39185)
I20260811 12:14:03.718497 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.711807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:60019 (local address 127.4.48.193:45871)
0811 12:14:03.711990 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.711994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.712511 (+ 517us) server_negotiation.cc:407] Connection header received
0811 12:14:03.712761 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.712766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.712862 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.712955 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:03.713652 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.714623 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.716675 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.717495 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.717612 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.717663 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.718089 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.718225 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 12:14:03.718525 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.712760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:47419 (local address 127.4.48.195:39185)
0811 12:14:03.712969 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.712975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.713209 (+ 234us) server_negotiation.cc:407] Connection header received
0811 12:14:03.713395 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.713399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.713467 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.713532 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:03.714016 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.714951 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.716847 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.717620 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.717768 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.717835 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.718181 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.718307 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:14:03.718549 4336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.711614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:45871 (local address 127.4.48.194:60019)
0811 12:14:03.712421 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.712441 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.712586 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.713045 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.713049 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.713078 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:03.713507 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.713519 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.714770 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.714775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.716479 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.716493 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.717682 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.717689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.717917 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.717947 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.718019 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.718118 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":5,"spinlock_wait_cycles":25856,"thread_start_us":450,"threads_started":1}
I20260811 12:14:03.718592 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.712534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:39185 (local address 127.4.48.194:47419)
0811 12:14:03.713114 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:03.713131 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:03.713271 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:03.713569 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:03.713573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:03.713600 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:03.713882 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.713890 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.715117 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.715122 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.716653 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:03.716669 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.717803 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.717808 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:03.717986 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.718015 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 12:14:03.718107 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 12:14:03.718225 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"mutex_wait_us":38,"thread_start_us":292,"threads_started":1}
I20260811 12:14:03.719621 3734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3118598e614437854f0dbf8ec4cb17" candidate_uuid: "901bca4f8b7448e08b430f978c50898a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eff668bbbef458b9fe90a2024be7a47" is_pre_election: true
I20260811 12:14:03.719621 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3118598e614437854f0dbf8ec4cb17" candidate_uuid: "901bca4f8b7448e08b430f978c50898a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" is_pre_election: true
I20260811 12:14:03.720243 4200 raft_consensus.cc:2454] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901bca4f8b7448e08b430f978c50898a in term 0.
I20260811 12:14:03.720243 3734 raft_consensus.cc:2454] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 901bca4f8b7448e08b430f978c50898a in term 0.
I20260811 12:14:03.720937 3856 leader_election.cc:302] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ebd31ecaea34273b8aca12e98c84c0e, 901bca4f8b7448e08b430f978c50898a; no voters:
I20260811 12:14:03.721297 4335 raft_consensus.cc:2781] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:14:03.721459 4335 raft_consensus.cc:482] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:14:03.721577 4335 raft_consensus.cc:3037] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:03.723699 4335 raft_consensus.cc:504] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.724261 4335 leader_election.cc:288] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [CANDIDATE]: Term 1 election: Requested vote from peers 7eff668bbbef458b9fe90a2024be7a47 (127.4.48.193:45871), 1ebd31ecaea34273b8aca12e98c84c0e (127.4.48.195:39185)
I20260811 12:14:03.724669 3734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3118598e614437854f0dbf8ec4cb17" candidate_uuid: "901bca4f8b7448e08b430f978c50898a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7eff668bbbef458b9fe90a2024be7a47"
I20260811 12:14:03.724777 4200 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee3118598e614437854f0dbf8ec4cb17" candidate_uuid: "901bca4f8b7448e08b430f978c50898a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ebd31ecaea34273b8aca12e98c84c0e"
I20260811 12:14:03.724851 3734 raft_consensus.cc:3037] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:03.724974 4200 raft_consensus.cc:3037] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:03.727202 3734 raft_consensus.cc:2454] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901bca4f8b7448e08b430f978c50898a in term 1.
I20260811 12:14:03.727253 4200 raft_consensus.cc:2454] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 901bca4f8b7448e08b430f978c50898a in term 1.
I20260811 12:14:03.727562 3856 leader_election.cc:302] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ebd31ecaea34273b8aca12e98c84c0e, 901bca4f8b7448e08b430f978c50898a; no voters:
I20260811 12:14:03.727831 4335 raft_consensus.cc:2781] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 1 FOLLOWER]: Leader election won for term 1
W20260811 12:14:03.727890 4275 tablet.cc:2369] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:03.728304 4335 raft_consensus.cc:686] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 1 LEADER]: Becoming Leader. State: Replica: 901bca4f8b7448e08b430f978c50898a, State: Running, Role: LEADER
I20260811 12:14:03.728811 4335 consensus_queue.cc:237] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } }
I20260811 12:14:03.733155 3474 catalog_manager.cc:5495] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a reported cstate change: term changed from 0 to 1, leader changed from <none> to 901bca4f8b7448e08b430f978c50898a (127.4.48.194). New cstate: current_term: 1 leader_uuid: "901bca4f8b7448e08b430f978c50898a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "901bca4f8b7448e08b430f978c50898a" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 45963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 } health_report { overall_health: UNKNOWN } } }
I20260811 12:14:03.772492 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.767093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.4.48.194:45963)
0811 12:14:03.767293 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.767299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.767318 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:14:03.767537 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.767541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.767627 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.767771 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:03.768169 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.768883 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.770286 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.770700 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.770808 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.771424 (+ 616us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:03.771939 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:03.772097 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.772231 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.772273 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:14:03.780290 3734 raft_consensus.cc:1261] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 1 FOLLOWER]: Refusing update from remote peer 901bca4f8b7448e08b430f978c50898a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:03.780305 4200 raft_consensus.cc:1261] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 1 FOLLOWER]: Refusing update from remote peer 901bca4f8b7448e08b430f978c50898a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:03.781212 4339 consensus_queue.cc:1035] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ebd31ecaea34273b8aca12e98c84c0e" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:03.781534 4335 consensus_queue.cc:1035] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7eff668bbbef458b9fe90a2024be7a47" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 45871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:14:03.782408 4056 tablet.cc:2369] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:03.805405 3473 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:03.805641 3473 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55342:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:03.809194 3473 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 397737968d664df0b49b68f6dad8863d: Sending DeleteTablet for 3 replicas of tablet ee3118598e614437854f0dbf8ec4cb17
I20260811 12:14:03.811197 4164 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3118598e614437854f0dbf8ec4cb17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:03 UTC) from {username='slave'} at 127.0.0.1:41218
I20260811 12:14:03.810629 3700 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3118598e614437854f0dbf8ec4cb17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:03 UTC) from {username='slave'} at 127.0.0.1:43156
I20260811 12:14:03.811995 3938 tablet_service.cc:1514] Processing DeleteTablet for tablet ee3118598e614437854f0dbf8ec4cb17 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:03 UTC) from {username='slave'} at 127.0.0.1:42644
I20260811 12:14:03.813184 4371 tablet_replica.cc:330] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: stopping tablet replica
I20260811 12:14:03.813714 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 3582
I20260811 12:14:03.813642 4371 raft_consensus.cc:2229] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:14:03.814293 4371 raft_consensus.cc:2258] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:03.816493 4371 ts_tablet_manager.cc:1905] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:03.823985 4371 ts_tablet_manager.cc:1918] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:03.824146 4371 log.cc:1196] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/ee3118598e614437854f0dbf8ec4cb17
I20260811 12:14:03.824707 4371 ts_tablet_manager.cc:1939] T ee3118598e614437854f0dbf8ec4cb17 P 901bca4f8b7448e08b430f978c50898a: Deleting consensus metadata
I20260811 12:14:03.826239 4368 tablet_replica.cc:330] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: stopping tablet replica
I20260811 12:14:03.826686 4368 raft_consensus.cc:2229] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:03.827292 4368 raft_consensus.cc:2258] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:03.828558 3449 catalog_manager.cc:4841] TS 901bca4f8b7448e08b430f978c50898a (127.4.48.194:45963): tablet ee3118598e614437854f0dbf8ec4cb17 (table test-table [id=0aa7f7ab506342658c658f09b1d76bfc]) successfully deleted
I20260811 12:14:03.829289 4368 ts_tablet_manager.cc:1905] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:03.836398 4368 ts_tablet_manager.cc:1918] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:03.836620 4368 log.cc:1196] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/ee3118598e614437854f0dbf8ec4cb17
I20260811 12:14:03.837249 4368 ts_tablet_manager.cc:1939] T ee3118598e614437854f0dbf8ec4cb17 P 7eff668bbbef458b9fe90a2024be7a47: Deleting consensus metadata
I20260811 12:14:03.839119 3447 catalog_manager.cc:4841] TS 7eff668bbbef458b9fe90a2024be7a47 (127.4.48.193:45871): tablet ee3118598e614437854f0dbf8ec4cb17 (table test-table [id=0aa7f7ab506342658c658f09b1d76bfc]) successfully deleted
I20260811 12:14:03.839515 4372 tablet_replica.cc:330] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: stopping tablet replica
I20260811 12:14:03.840003 4372 raft_consensus.cc:2229] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:03.840497 4372 raft_consensus.cc:2258] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:03.842465 4372 ts_tablet_manager.cc:1905] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:03.843995 4306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.814985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.4.48.193:45871)
0811 12:14:03.815173 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.815180 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.835142 (+ 19962us) server_negotiation.cc:407] Connection header received
0811 12:14:03.835429 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.835435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.835543 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.835618 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:03.836384 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.837610 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.840302 (+ 2692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.840944 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.841085 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.842342 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:03.842353 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:03.842371 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:03.843099 (+ 728us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:03.843160 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:03.843184 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:03.843309 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.843474 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.843587 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:14:03.844691 3809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:03.848434 4372 ts_tablet_manager.cc:1918] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:03.848589 4372 log.cc:1196] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/ee3118598e614437854f0dbf8ec4cb17
I20260811 12:14:03.849082 4372 ts_tablet_manager.cc:1939] T ee3118598e614437854f0dbf8ec4cb17 P 1ebd31ecaea34273b8aca12e98c84c0e: Deleting consensus metadata
I20260811 12:14:03.850849 3447 catalog_manager.cc:4841] TS 1ebd31ecaea34273b8aca12e98c84c0e (127.4.48.195:39185): tablet ee3118598e614437854f0dbf8ec4cb17 (table test-table [id=0aa7f7ab506342658c658f09b1d76bfc]) successfully deleted
I20260811 12:14:03.932631 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 3582
I20260811 12:14:03.966886 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 3822
I20260811 12:14:03.974315 4308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:03.967619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42660 (local address 127.4.48.194:45963)
0811 12:14:03.967812 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:03.967816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:03.967876 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:14:03.968067 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:03.968071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:03.968156 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:03.968232 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:03.968687 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.969470 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.971317 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:03.971782 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:03.971873 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:03.972961 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:03.972971 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:03.972986 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:03.973597 (+ 611us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:03.973652 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:03.973671 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:03.973751 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:03.973899 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 12:14:03.973976 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 12:14:03.975042 4047 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:04.093858 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 3822
I20260811 12:14:04.118115 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4058
I20260811 12:14:04.128432 4311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:04.118885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41234 (local address 127.4.48.195:39185)
0811 12:14:04.119072 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:04.119078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:04.119161 (+ 83us) server_negotiation.cc:407] Connection header received
0811 12:14:04.119350 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:04.119356 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:04.119461 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:04.119549 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:04.120016 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:04.120732 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:04.122108 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:04.122514 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:04.122624 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:04.127080 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:04.127089 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:04.127105 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:04.127658 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:04.127747 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:04.127769 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:04.127870 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:04.127998 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 12:14:04.128111 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:14:04.129051 4269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:04.229171 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4058
I20260811 12:14:04.252095 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 3417
I20260811 12:14:04.253145 3531 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:04.358738 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 3417
2026-08-11T12:14:04Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2817 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:04 dist-test-slave-nj21 krb5kdc[4385](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:04 dist-test-slave-nj21 krb5kdc[4385](info): set up 2 sockets
Aug 11 12:14:04 dist-test-slave-nj21 krb5kdc[4385](info): commencing operation
krb5kdc: starting...
W20260811 12:14:05.602223 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.003s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:05 dist-test-slave-nj21 krb5kdc[4385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:05.872387 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45613
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:45613
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:06.196734 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:06.197010 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:06.197099 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:06.204071 4408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:06.204165 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:06.204218 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:06.204273 4408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:06.204320 4408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:06.214007 4408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:45613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:45613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:06.215270 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:06.216568 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:06.227726 4418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:06.227792 4423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:06.227823 4420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:06.228500 4408 server_base.cc:1034] running on GCE node
I20260811 12:14:06.229254 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:06.230544 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:06.231779 4408 hybrid_clock.cc:648] HybridClock initialized: now 1786450446231682 us; error 56 us; skew 500 ppm
I20260811 12:14:06.232105 4408 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:06 dist-test-slave-nj21 krb5kdc[4385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:06.238189 4408 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:06.239555 4408 webserver.cc:460] Webserver started at http://127.4.48.254:35911/ using document root <none> and password file <none>
I20260811 12:14:06.240231 4408 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:06.240329 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:06.240578 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:06.243448 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "af9d094804904a52b0c5d2f2252efa33"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "55845ffec97aaa393522846cb7e1187b"
server_key_iv: "323523faad2b5ded2be6df7d0a1a74fb"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.244156 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "af9d094804904a52b0c5d2f2252efa33"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "55845ffec97aaa393522846cb7e1187b"
server_key_iv: "323523faad2b5ded2be6df7d0a1a74fb"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.248924 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:14:06.252082 4440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:06.253383 4408 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:06.253546 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "af9d094804904a52b0c5d2f2252efa33"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "55845ffec97aaa393522846cb7e1187b"
server_key_iv: "323523faad2b5ded2be6df7d0a1a74fb"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.253726 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:06.277097 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:06.280723 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:06.280979 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:06.297964 4408 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:45613
I20260811 12:14:06.298000 4539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:45613 every 8 connection(s)
I20260811 12:14:06.299253 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:06.302860 4540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:06.303915 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4408
I20260811 12:14:06.304065 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:06.304499 4291 external_mini_cluster.cc:1428] Setting key 7fae75d4e35080131f08ae469dcb3251
I20260811 12:14:06.316845 4540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: Bootstrap starting.
Aug 11 12:14:06 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:06.319759 4540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:06.320878 4540 log.cc:824] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:06.323571 4540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: No bootstrap required, opened a new log
I20260811 12:14:06.326812 4548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:06.306222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54684 (local address 127.4.48.254:45613)
0811 12:14:06.306981 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:06.306996 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:06.307049 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:14:06.309224 (+ 2175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:06.309254 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:06.309751 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:06.310201 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:06.311559 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.313531 (+ 1972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.314811 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.315896 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.316072 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:06.320004 (+ 3932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:06.320012 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:06.320028 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:06.320071 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:06.322985 (+ 2914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:06.323643 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:06.323649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:06.323655 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:06.323869 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:06.324228 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:06.324234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:06.324237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:06.324723 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:06.325022 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:06.325490 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 12:14:06.325872 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":515,"thread_start_us":296,"threads_started":1}
I20260811 12:14:06.331771 4540 raft_consensus.cc:348] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } }
I20260811 12:14:06.332119 4540 raft_consensus.cc:374] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:06.332207 4540 raft_consensus.cc:729] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af9d094804904a52b0c5d2f2252efa33, State: Initialized, Role: FOLLOWER
I20260811 12:14:06.332712 4540 consensus_queue.cc:260] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } }
I20260811 12:14:06.332942 4540 raft_consensus.cc:388] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:06.333076 4540 raft_consensus.cc:482] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:06.333285 4540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:06.335606 4540 raft_consensus.cc:504] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } }
I20260811 12:14:06.336099 4540 leader_election.cc:302] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af9d094804904a52b0c5d2f2252efa33; no voters:
I20260811 12:14:06.336652 4540 leader_election.cc:288] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:06.336798 4550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:06.337404 4550 raft_consensus.cc:686] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [term 1 LEADER]: Becoming Leader. State: Replica: af9d094804904a52b0c5d2f2252efa33, State: Running, Role: LEADER
I20260811 12:14:06.337915 4550 consensus_queue.cc:237] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } }
I20260811 12:14:06.338505 4540 sys_catalog.cc:564] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:06.340705 4552 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af9d094804904a52b0c5d2f2252efa33. Latest consensus state: current_term: 1 leader_uuid: "af9d094804904a52b0c5d2f2252efa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } } }
I20260811 12:14:06.340780 4551 sys_catalog.cc:455] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af9d094804904a52b0c5d2f2252efa33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af9d094804904a52b0c5d2f2252efa33" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 45613 } } }
I20260811 12:14:06.341151 4552 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:06.341396 4551 sys_catalog.cc:458] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:06.342182 4556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:06.347419 4556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:06.356376 4556 catalog_manager.cc:1347] Generated new cluster ID: cbc28366ed114411bf920c4dad581d40
I20260811 12:14:06.356470 4556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:06.381191 4556 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:06.382499 4556 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:06.392673 4556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: Generated new TSK 0
I20260811 12:14:06.393443 4556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.4.48.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
Entry for principal kudu/127.4.48.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab.
I20260811 12:14:06.464126 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:06.793905 4577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:06.794159 4577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:06.794250 4577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:06.801191 4577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:06.801286 4577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:06.801400 4577 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:14:06.810643 4577 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.193.keytab
--principal=kudu/127.4.48.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:06.811815 4577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:06.813030 4577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:06.826972 4591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:06.826951 4587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:06.829207 4586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:06.830188 4577 server_base.cc:1034] running on GCE node
I20260811 12:14:06.830765 4577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:06.831700 4577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:06.832927 4577 hybrid_clock.cc:648] HybridClock initialized: now 1786450446832870 us; error 45 us; skew 500 ppm
I20260811 12:14:06.833226 4577 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:06 dist-test-slave-nj21 krb5kdc[4385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:06.836375 4577 init.cc:377] Logged in from keytab as kudu/127.4.48.193@KRBTEST.COM (short username kudu)
I20260811 12:14:06.837687 4577 webserver.cc:460] Webserver started at http://127.4.48.193:35989/ using document root <none> and password file <none>
I20260811 12:14:06.838299 4577 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:06.838402 4577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:06.838703 4577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:06.841588 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "26f52fd8e28140c994e393d0e7af2188"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "cb6eb82159ddd6f9f37dc4a18602c6d8"
server_key_iv: "22e1ffde012bab61f647011d26e2ffb6"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.842422 4577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "26f52fd8e28140c994e393d0e7af2188"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "cb6eb82159ddd6f9f37dc4a18602c6d8"
server_key_iv: "22e1ffde012bab61f647011d26e2ffb6"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.847230 4577 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.005s
I20260811 12:14:06.850498 4605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:06.851765 4577 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 12:14:06.851928 4577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "26f52fd8e28140c994e393d0e7af2188"
format_stamp: "Formatted at 2026-08-11 12:14:06 on dist-test-slave-nj21"
server_key: "cb6eb82159ddd6f9f37dc4a18602c6d8"
server_key_iv: "22e1ffde012bab61f647011d26e2ffb6"
server_key_version: "encryptionkey@0"
I20260811 12:14:06.852103 4577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:06.876693 4577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:06.880235 4577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:06.880493 4577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:06.881371 4577 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:06.882596 4577 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:06.882668 4577 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:06.882782 4577 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:06.882838 4577 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:06.912421 4577 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:34425
I20260811 12:14:06.912470 4824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:34425 every 8 connection(s)
I20260811 12:14:06.913638 4577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:14:06.919895 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4577
I20260811 12:14:06.920055 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:14:06.920462 4291 external_mini_cluster.cc:1428] Setting key e144920b73f7fcd3d957ee8bac28ecf2
Aug 11 12:14:06 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450446, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.193@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:06.933769 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:06.915999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:52893 (local address 127.4.48.254:45613)
0811 12:14:06.916442 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:06.916447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:06.917226 (+ 779us) server_negotiation.cc:407] Connection header received
0811 12:14:06.919094 (+ 1868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:06.919100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:06.919179 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:06.919293 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:06.920870 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.921832 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.924000 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.924411 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.924511 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:06.928374 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:06.928378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:06.928381 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:06.928413 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:06.930802 (+ 2389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:06.931590 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:06.931593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:06.931595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:06.931745 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:06.932192 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:06.932195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:06.932197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:06.932467 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:06.932578 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:06.933364 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 12:14:06.933528 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":222,"threads_started":1}
I20260811 12:14:06.935016 4830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:06.916403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45613 (local address 127.4.48.193:52893)
0811 12:14:06.917073 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:06.917109 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:06.918818 (+ 1709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:06.919525 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:06.919533 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:06.920102 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:06.920695 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:06.920715 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.921999 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.922004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:06.923849 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:06.923866 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:06.924642 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:06.924650 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:06.924937 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:06.925613 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:06.925639 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:06.928161 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:06.930948 (+ 2787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:06.930958 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:06.930973 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:06.931490 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:06.931842 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:06.931847 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:06.931850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:06.932094 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:06.932586 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:06.932593 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:06.932999 (+ 406us) client_negotiation.cc:769] Sending connection context
0811 12:14:06.933275 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 12:14:06.933748 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":259,"threads_started":1}
I20260811 12:14:06.936582 4826 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45613
I20260811 12:14:06.936923 4826 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:06.937680 4826 heartbeater.cc:511] Master 127.4.48.254:45613 requested a full tablet report, sending...
I20260811 12:14:06.939926 4469 ts_manager.cc:194] Registered new tserver with Master: 26f52fd8e28140c994e393d0e7af2188 (127.4.48.193:34425)
I20260811 12:14:06.941802 4469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.193@KRBTEST.COM'} at 127.4.48.193:52893
WARNING: no policy specified for kudu/127.4.48.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
Entry for principal kudu/127.4.48.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab.
I20260811 12:14:06.985801 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:07.308573 4835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:07.308832 4835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:07.308980 4835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:07.315917 4835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:07.316020 4835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:07.316162 4835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:14:07.325332 4835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.194.keytab
--principal=kudu/127.4.48.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:07.326524 4835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:07.327802 4835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:07.340384 4846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:07.340332 4849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:07.340418 4844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:07.341267 4835 server_base.cc:1034] running on GCE node
I20260811 12:14:07.341770 4835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:07.342608 4835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:07.343755 4835 hybrid_clock.cc:648] HybridClock initialized: now 1786450447343711 us; error 31 us; skew 500 ppm
I20260811 12:14:07.344035 4835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:07 dist-test-slave-nj21 krb5kdc[4385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:07.346979 4835 init.cc:377] Logged in from keytab as kudu/127.4.48.194@KRBTEST.COM (short username kudu)
I20260811 12:14:07.348351 4835 webserver.cc:460] Webserver started at http://127.4.48.194:35873/ using document root <none> and password file <none>
I20260811 12:14:07.348941 4835 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:07.349040 4835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:07.349305 4835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:07.352070 4835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "878fccff21534d469d58862ff21bd5a0"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "2bde81fa982e511e503070a687433343"
server_key_iv: "2240e8b2028716c9f047143d8c15b146"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.352867 4835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "878fccff21534d469d58862ff21bd5a0"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "2bde81fa982e511e503070a687433343"
server_key_iv: "2240e8b2028716c9f047143d8c15b146"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.357834 4835 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:14:07.361151 4863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.362371 4835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:14:07.362519 4835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "878fccff21534d469d58862ff21bd5a0"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "2bde81fa982e511e503070a687433343"
server_key_iv: "2240e8b2028716c9f047143d8c15b146"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.362689 4835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:07.382066 4835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:07.386031 4835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:07.386286 4835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:07.387183 4835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:07.388550 4835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:07.388628 4835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.388746 4835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:07.388799 4835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.418486 4835 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:39127
I20260811 12:14:07.418586 5061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:39127 every 8 connection(s)
I20260811 12:14:07.419731 4835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:14:07.428974 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 4835
I20260811 12:14:07.429160 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:14:07.429787 4291 external_mini_cluster.cc:1428] Setting key 01f4abd0b2047b347a1a5a8cad691969
Aug 11 12:14:07 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.194@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:07.439814 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.422189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:33737 (local address 127.4.48.254:45613)
0811 12:14:07.422358 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:07.422362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:07.423438 (+ 1076us) server_negotiation.cc:407] Connection header received
0811 12:14:07.425403 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:07.425410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:07.425489 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:07.425578 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:07.427231 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.427981 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.430016 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.430541 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.430657 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:07.433963 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:07.433970 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:07.433974 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:07.434015 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:07.437010 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.437665 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.437670 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.437673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.437822 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.438190 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.438194 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.438197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.438511 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:07.438666 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:07.439247 (+ 581us) server_negotiation.cc:299] Negotiation successful
0811 12:14:07.439473 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:14:07.440644 5064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.422614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45613 (local address 127.4.48.194:33737)
0811 12:14:07.423288 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:07.423327 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:07.425133 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:07.425814 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:07.425821 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:07.426351 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:07.427062 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:07.427085 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.428141 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.428147 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:07.429860 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:07.429876 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.430778 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.430785 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:07.431001 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:07.431646 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:07.431701 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:07.433741 (+ 2040us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:07.437133 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:07.437140 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:07.437151 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:07.437556 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:07.437916 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:07.437919 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:07.437921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:07.438092 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:07.438627 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:07.438633 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:07.438919 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 12:14:07.439137 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 12:14:07.439521 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":261,"threads_started":1}
I20260811 12:14:07.442025 5062 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45613
I20260811 12:14:07.442363 5062 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:07.443156 5062 heartbeater.cc:511] Master 127.4.48.254:45613 requested a full tablet report, sending...
I20260811 12:14:07.444785 4469 ts_manager.cc:194] Registered new tserver with Master: 878fccff21534d469d58862ff21bd5a0 (127.4.48.194:39127)
I20260811 12:14:07.445637 4469 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.194@KRBTEST.COM'} at 127.4.48.194:33737
WARNING: no policy specified for kudu/127.4.48.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
Entry for principal kudu/127.4.48.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab.
I20260811 12:14:07.492100 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:34399
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:07.819175 5068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:07.819459 5068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:07.819605 5068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:07.827834 5068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:07.827958 5068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:07.828116 5068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:14:07.837630 5068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:34399
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.195.keytab
--principal=kudu/127.4.48.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:45613
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:07.838932 5068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:07.840260 5068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:07.852644 5077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:07.852582 5083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:07.852609 5079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:07.853662 5068 server_base.cc:1034] running on GCE node
I20260811 12:14:07.854193 5068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:07.855042 5068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:07.856246 5068 hybrid_clock.cc:648] HybridClock initialized: now 1786450447856205 us; error 32 us; skew 500 ppm
I20260811 12:14:07.856554 5068 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:07 dist-test-slave-nj21 krb5kdc[4385](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:07.859501 5068 init.cc:377] Logged in from keytab as kudu/127.4.48.195@KRBTEST.COM (short username kudu)
I20260811 12:14:07.860810 5068 webserver.cc:460] Webserver started at http://127.4.48.195:42613/ using document root <none> and password file <none>
I20260811 12:14:07.861433 5068 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:07.861539 5068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:07.861831 5068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:07.864598 5068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "dc8acd828ab744dd819869041dfc1ded"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "aa9ad3b558de7ee060aec1a605caee86"
server_key_iv: "55fa4997fe1950296d9d76e258012094"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.865322 5068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "dc8acd828ab744dd819869041dfc1ded"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "aa9ad3b558de7ee060aec1a605caee86"
server_key_iv: "55fa4997fe1950296d9d76e258012094"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.870307 5068 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:14:07.873471 5093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.874660 5068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:14:07.874802 5068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "dc8acd828ab744dd819869041dfc1ded"
format_stamp: "Formatted at 2026-08-11 12:14:07 on dist-test-slave-nj21"
server_key: "aa9ad3b558de7ee060aec1a605caee86"
server_key_iv: "55fa4997fe1950296d9d76e258012094"
server_key_version: "encryptionkey@0"
I20260811 12:14:07.874969 5068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:07.893519 5068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:07.897027 5068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:07.897285 5068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:07.898156 5068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:07.899371 5068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:07.899441 5068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.899561 5068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:07.899611 5068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:07.929297 5068 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:32783
I20260811 12:14:07.929335 5299 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:32783 every 8 connection(s)
I20260811 12:14:07.930497 5068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:14:07.936285 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5068
I20260811 12:14:07.936452 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:14:07.936866 4291 external_mini_cluster.cc:1428] Setting key 80b0f99f72f454ca4a84eb8c2fe0c4ac
Aug 11 12:14:07 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450447, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.195@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:07.944631 4826 heartbeater.cc:503] Master 127.4.48.254:45613 was elected leader, sending a full tablet report...
I20260811 12:14:07.949357 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.932891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:43297 (local address 127.4.48.254:45613)
0811 12:14:07.933078 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:07.933083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:07.934146 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 12:14:07.936008 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:07.936014 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:07.936087 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:07.936176 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:07.937776 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.938793 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.940481 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.940863 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.940967 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:07.943990 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:07.943994 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:07.943998 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:07.944030 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:07.946834 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.947441 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.947444 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.947446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.947563 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.948011 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.948014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.948016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.948277 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:07.948397 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:07.948948 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 12:14:07.949125 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:14:07.950352 5302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.933310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:45613 (local address 127.4.48.195:43297)
0811 12:14:07.933986 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:07.934028 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:07.935703 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:07.936430 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:07.936441 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:07.936977 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:07.937577 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:07.937602 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.938939 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.938944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:07.940348 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:07.940360 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.941039 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.941044 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:07.941222 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:07.941767 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:07.941801 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:07.943796 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:07.946955 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:07.946961 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:07.946971 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:07.947351 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:07.947705 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:07.947710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:07.947714 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:07.947913 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:07.948376 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:07.948381 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:07.948651 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 12:14:07.948870 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 12:14:07.949200 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":256,"threads_started":1}
I20260811 12:14:07.951831 5300 heartbeater.cc:347] Connected to a master server at 127.4.48.254:45613
I20260811 12:14:07.952112 5300 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:07.952775 5300 heartbeater.cc:511] Master 127.4.48.254:45613 requested a full tablet report, sending...
I20260811 12:14:07.954032 4467 ts_manager.cc:194] Registered new tserver with Master: dc8acd828ab744dd819869041dfc1ded (127.4.48.195:32783)
I20260811 12:14:07.954813 4467 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.4.48.195@KRBTEST.COM'} at 127.4.48.195:43297
I20260811 12:14:07.963594 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:14:07.978727 4828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.967441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.4.48.254:45613)
0811 12:14:07.967620 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:07.967627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:07.968035 (+ 408us) server_negotiation.cc:407] Connection header received
0811 12:14:07.968149 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:07.968153 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:07.968207 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:07.968277 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:07.969535 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.970221 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.971406 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:07.971842 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:07.971940 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:07.973988 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:07.973993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:07.973996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:07.974024 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:07.976395 (+ 2371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.976910 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.976913 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.976915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.977033 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:07.977288 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:07.977291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:07.977293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:07.977603 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:07.977866 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:07.978232 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 12:14:07.978430 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:14:07.982108 4467 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:14:07.988798 4467 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:14:08.009925 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:07.999629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:34425 (local address 127.0.0.1:45554)
0811 12:14:08.000354 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:08.000400 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:08.000567 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:08.001501 (+ 934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:08.001508 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:08.001535 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:08.001917 (+ 382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.001931 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.004178 (+ 2247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.004184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.006555 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.006572 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.008908 (+ 2336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.008915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.009131 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.009182 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 12:14:08.009310 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:14:08.009513 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":359,"threads_started":1}
I20260811 12:14:08.011229 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.000057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.4.48.193:34425)
0811 12:14:08.000668 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.000676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.000705 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:14:08.000804 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.000812 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.001069 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.001267 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:08.002143 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.004011 (+ 1868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.006762 (+ 2751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.008350 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.008495 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.008746 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.010693 (+ 1947us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.010891 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":304,"threads_started":1}
I20260811 12:14:08.015946 4704 tablet_service.cc:1467] Processing CreateTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 (DEFAULT_TABLE table=test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:08.017359 4704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc36382a6bb4d2e9c1f9192477fd868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:08.018412 5316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.001279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:32783 (local address 127.0.0.1:54780)
0811 12:14:08.010406 (+ 9127us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:08.010425 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:08.010530 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:08.012631 (+ 2101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:08.012636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:08.012660 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:08.012819 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.012830 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.014756 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.014761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.016108 (+ 1347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.016120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.017647 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.017653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.017854 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.017899 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 12:14:08.017996 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:14:08.018131 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9014,"thread_start_us":10045,"threads_started":1}
I20260811 12:14:08.019556 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.001801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.4.48.195:32783)
0811 12:14:08.012136 (+ 10335us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.012144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.012176 (+ 32us) server_negotiation.cc:407] Connection header received
0811 12:14:08.012271 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.012280 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.012461 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.012630 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:08.013002 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.014592 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.016247 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.017486 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.017611 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.017815 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.019092 (+ 1277us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.019248 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10177,"thread_start_us":10032,"threads_started":1}
I20260811 12:14:08.020633 5319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.000720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39127 (local address 127.0.0.1:47502)
0811 12:14:08.001221 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:08.001238 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:08.001340 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:08.013918 (+ 12578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:08.013923 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:08.013950 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:08.014301 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.014312 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.016614 (+ 2302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.016619 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.018490 (+ 1871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.018503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.019921 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.019928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.020089 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.020119 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:14:08.020216 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 12:14:08.020338 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":233,"threads_started":1}
I20260811 12:14:08.020769 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.000998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47502 (local address 127.4.48.194:39127)
0811 12:14:08.013365 (+ 12367us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.013372 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.013401 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:14:08.013495 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.013503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.013707 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.013893 (+ 186us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:08.014486 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.016465 (+ 1979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.018661 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.019731 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.019860 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.020045 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.020291 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.020461 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12201,"thread_start_us":13061,"threads_started":1}
I20260811 12:14:08.023404 5193 tablet_service.cc:1467] Processing CreateTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 (DEFAULT_TABLE table=test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:08.024564 5193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc36382a6bb4d2e9c1f9192477fd868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:08.024487 4951 tablet_service.cc:1467] Processing CreateTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 (DEFAULT_TABLE table=test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:08.025806 4951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dcc36382a6bb4d2e9c1f9192477fd868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:08.031723 5324 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Bootstrap starting.
I20260811 12:14:08.035475 5324 tablet_bootstrap.cc:654] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:08.036478 5325 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Bootstrap starting.
I20260811 12:14:08.036837 5324 log.cc:824] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:08.039489 5330 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Bootstrap starting.
I20260811 12:14:08.039615 5324 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: No bootstrap required, opened a new log
I20260811 12:14:08.039943 5324 ts_tablet_manager.cc:1397] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.004s
I20260811 12:14:08.040331 5325 tablet_bootstrap.cc:654] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:08.041756 5325 log.cc:824] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:08.043002 5330 tablet_bootstrap.cc:654] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:08.044163 5330 log.cc:824] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:08.044395 5325 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: No bootstrap required, opened a new log
I20260811 12:14:08.044664 5325 ts_tablet_manager.cc:1397] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 12:14:08.047160 5330 tablet_bootstrap.cc:492] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: No bootstrap required, opened a new log
I20260811 12:14:08.047484 5330 ts_tablet_manager.cc:1397] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 12:14:08.049677 5324 raft_consensus.cc:348] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.050107 5324 raft_consensus.cc:374] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:08.050209 5324 raft_consensus.cc:729] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f52fd8e28140c994e393d0e7af2188, State: Initialized, Role: FOLLOWER
I20260811 12:14:08.050819 5324 consensus_queue.cc:260] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.052551 5324 ts_tablet_manager.cc:1428] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260811 12:14:08.053862 5325 raft_consensus.cc:348] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.054304 5325 raft_consensus.cc:374] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:08.054468 5325 raft_consensus.cc:729] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc8acd828ab744dd819869041dfc1ded, State: Initialized, Role: FOLLOWER
I20260811 12:14:08.055099 5325 consensus_queue.cc:260] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.056670 5325 ts_tablet_manager.cc:1428] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Time spent starting tablet: real 0.012s user 0.009s sys 0.001s
I20260811 12:14:08.057179 5300 heartbeater.cc:503] Master 127.4.48.254:45613 was elected leader, sending a full tablet report...
I20260811 12:14:08.057543 5330 raft_consensus.cc:348] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.057977 5330 raft_consensus.cc:374] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:08.058130 5330 raft_consensus.cc:729] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 878fccff21534d469d58862ff21bd5a0, State: Initialized, Role: FOLLOWER
I20260811 12:14:08.058686 5330 consensus_queue.cc:260] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.059998 5062 heartbeater.cc:503] Master 127.4.48.254:45613 was elected leader, sending a full tablet report...
I20260811 12:14:08.060530 5330 ts_tablet_manager.cc:1428] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
W20260811 12:14:08.165957 4827 tablet.cc:2369] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:14:08.171475 5063 tablet.cc:2369] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:08.175637 5336 raft_consensus.cc:482] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:14:08.175940 5336 raft_consensus.cc:504] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.177536 5336 leader_election.cc:288] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 878fccff21534d469d58862ff21bd5a0 (127.4.48.194:39127), dc8acd828ab744dd819869041dfc1ded (127.4.48.195:32783)
W20260811 12:14:08.182013 5301 tablet.cc:2369] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:08.184237 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.177952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:45751 (local address 127.4.48.194:39127)
0811 12:14:08.178146 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.178153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.178490 (+ 337us) server_negotiation.cc:407] Connection header received
0811 12:14:08.178753 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.178759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.178840 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.178946 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:08.179661 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.180406 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.182376 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.183251 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.183369 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.183426 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.183837 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.183987 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:14:08.184255 5342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.177749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39127 (local address 127.4.48.193:45751)
0811 12:14:08.178372 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:08.178405 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:08.178554 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:08.179037 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:08.179042 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:08.179070 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:08.179523 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.179533 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.180536 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.180542 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.182208 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.182224 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.183456 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.183463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.183627 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.183663 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 12:14:08.183780 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:14:08.183901 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":23,"spinlock_wait_cycles":41216,"thread_start_us":301,"threads_started":1}
I20260811 12:14:08.184530 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.178717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:39035 (local address 127.4.48.195:32783)
0811 12:14:08.178916 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.178921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.179248 (+ 327us) server_negotiation.cc:407] Connection header received
0811 12:14:08.179412 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.179417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.179476 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.179549 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:08.180114 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.180818 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.182680 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.183547 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.183726 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.183795 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.184173 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.184301 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 12:14:08.184705 5343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.178494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:32783 (local address 127.4.48.193:39035)
0811 12:14:08.179128 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:08.179151 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:08.179284 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:08.179583 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:08.179587 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:08.179610 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:08.179961 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.179971 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.180962 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.180966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.182506 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:08.182520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.183800 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.183808 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:08.183986 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.184023 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 12:14:08.184117 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 12:14:08.184236 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"mutex_wait_us":26,"thread_start_us":311,"threads_started":1}
I20260811 12:14:08.185534 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc36382a6bb4d2e9c1f9192477fd868" candidate_uuid: "26f52fd8e28140c994e393d0e7af2188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878fccff21534d469d58862ff21bd5a0" is_pre_election: true
I20260811 12:14:08.185873 5218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc36382a6bb4d2e9c1f9192477fd868" candidate_uuid: "26f52fd8e28140c994e393d0e7af2188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8acd828ab744dd819869041dfc1ded" is_pre_election: true
I20260811 12:14:08.186009 4977 raft_consensus.cc:2454] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f52fd8e28140c994e393d0e7af2188 in term 0.
I20260811 12:14:08.186308 5218 raft_consensus.cc:2454] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26f52fd8e28140c994e393d0e7af2188 in term 0.
I20260811 12:14:08.186636 4607 leader_election.cc:302] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f52fd8e28140c994e393d0e7af2188, 878fccff21534d469d58862ff21bd5a0; no voters:
I20260811 12:14:08.187052 5336 raft_consensus.cc:2781] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:14:08.187204 5336 raft_consensus.cc:482] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:14:08.187323 5336 raft_consensus.cc:3037] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:08.189524 5336 raft_consensus.cc:504] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.190069 5336 leader_election.cc:288] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [CANDIDATE]: Term 1 election: Requested vote from peers 878fccff21534d469d58862ff21bd5a0 (127.4.48.194:39127), dc8acd828ab744dd819869041dfc1ded (127.4.48.195:32783)
I20260811 12:14:08.190466 4977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc36382a6bb4d2e9c1f9192477fd868" candidate_uuid: "26f52fd8e28140c994e393d0e7af2188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "878fccff21534d469d58862ff21bd5a0"
I20260811 12:14:08.190554 5218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dcc36382a6bb4d2e9c1f9192477fd868" candidate_uuid: "26f52fd8e28140c994e393d0e7af2188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc8acd828ab744dd819869041dfc1ded"
I20260811 12:14:08.190645 4977 raft_consensus.cc:3037] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:08.190734 5218 raft_consensus.cc:3037] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:08.192811 4977 raft_consensus.cc:2454] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f52fd8e28140c994e393d0e7af2188 in term 1.
I20260811 12:14:08.192919 5218 raft_consensus.cc:2454] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26f52fd8e28140c994e393d0e7af2188 in term 1.
I20260811 12:14:08.193193 4607 leader_election.cc:302] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26f52fd8e28140c994e393d0e7af2188, 878fccff21534d469d58862ff21bd5a0; no voters:
I20260811 12:14:08.193434 5336 raft_consensus.cc:2781] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:08.193835 5336 raft_consensus.cc:686] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 1 LEADER]: Becoming Leader. State: Replica: 26f52fd8e28140c994e393d0e7af2188, State: Running, Role: LEADER
I20260811 12:14:08.194278 5336 consensus_queue.cc:237] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } }
I20260811 12:14:08.198865 4469 catalog_manager.cc:5495] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26f52fd8e28140c994e393d0e7af2188 (127.4.48.193). New cstate: current_term: 1 leader_uuid: "26f52fd8e28140c994e393d0e7af2188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26f52fd8e28140c994e393d0e7af2188" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 34425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 } health_report { overall_health: UNKNOWN } } }
I20260811 12:14:08.254909 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.250199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45566 (local address 127.4.48.193:34425)
0811 12:14:08.250427 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.250432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.250449 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:14:08.250546 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.250549 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.250627 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.250750 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:08.251048 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.251802 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.253046 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.253421 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.253506 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.253963 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:08.254447 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:08.254553 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.254588 (+ 35us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.254652 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 12:14:08.262836 5218 raft_consensus.cc:1261] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 1 FOLLOWER]: Refusing update from remote peer 26f52fd8e28140c994e393d0e7af2188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:08.262866 4977 raft_consensus.cc:1261] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 1 FOLLOWER]: Refusing update from remote peer 26f52fd8e28140c994e393d0e7af2188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:08.263727 5344 consensus_queue.cc:1035] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc8acd828ab744dd819869041dfc1ded" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:08.264092 5336 consensus_queue.cc:1035] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "878fccff21534d469d58862ff21bd5a0" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39127 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:08.287544 4467 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:08.287865 4467 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:08.291301 4467 catalog_manager.cc:5782] T 00000000000000000000000000000000 P af9d094804904a52b0c5d2f2252efa33: Sending DeleteTablet for 3 replicas of tablet dcc36382a6bb4d2e9c1f9192477fd868
I20260811 12:14:08.292327 4704 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:08 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 12:14:08.292488 5193 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:08 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:54780
I20260811 12:14:08.293164 4951 tablet_service.cc:1514] Processing DeleteTablet for tablet dcc36382a6bb4d2e9c1f9192477fd868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:08 UTC) from {username='kudu', principal='kudu/127.4.48.254@KRBTEST.COM'} at 127.0.0.1:47502
I20260811 12:14:08.293614 5382 tablet_replica.cc:330] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: stopping tablet replica
I20260811 12:14:08.294086 5382 raft_consensus.cc:2229] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:08.294417 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4577
I20260811 12:14:08.294610 5382 raft_consensus.cc:2258] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:08.297469 5380 tablet_replica.cc:330] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: stopping tablet replica
I20260811 12:14:08.297926 5380 raft_consensus.cc:2229] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:14:08.298625 5380 raft_consensus.cc:2258] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:08.301023 5380 ts_tablet_manager.cc:1905] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:08.304165 5382 ts_tablet_manager.cc:1905] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:08.306469 5386 tablet_replica.cc:330] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: stopping tablet replica
I20260811 12:14:08.306846 5386 raft_consensus.cc:2229] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:08.307307 5386 raft_consensus.cc:2258] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 12:14:08 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.193@KRBTEST.COM
I20260811 12:14:08.312187 5380 ts_tablet_manager.cc:1918] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:08.312435 5380 log.cc:1196] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-0/wal/wals/dcc36382a6bb4d2e9c1f9192477fd868
I20260811 12:14:08.313062 5380 ts_tablet_manager.cc:1939] T dcc36382a6bb4d2e9c1f9192477fd868 P 26f52fd8e28140c994e393d0e7af2188: Deleting consensus metadata
I20260811 12:14:08.315157 4441 catalog_manager.cc:4841] TS 26f52fd8e28140c994e393d0e7af2188 (127.4.48.193:34425): tablet dcc36382a6bb4d2e9c1f9192477fd868 (table test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]) successfully deleted
I20260811 12:14:08.315629 5382 ts_tablet_manager.cc:1918] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:08.315929 5382 log.cc:1196] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/dcc36382a6bb4d2e9c1f9192477fd868
I20260811 12:14:08.316488 5382 ts_tablet_manager.cc:1939] T dcc36382a6bb4d2e9c1f9192477fd868 P dc8acd828ab744dd819869041dfc1ded: Deleting consensus metadata
I20260811 12:14:08.318622 5386 ts_tablet_manager.cc:1905] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:08.323280 4441 catalog_manager.cc:4841] TS dc8acd828ab744dd819869041dfc1ded (127.4.48.195:32783): tablet dcc36382a6bb4d2e9c1f9192477fd868 (table test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]) successfully deleted
I20260811 12:14:08.324234 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.296565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.4.48.193:34425)
0811 12:14:08.296787 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.296795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.297478 (+ 683us) server_negotiation.cc:407] Connection header received
0811 12:14:08.297728 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.297734 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.297805 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.297874 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:08.299584 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.300730 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.304368 (+ 3638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.305062 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.305150 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.316818 (+ 11668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:08.316831 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:08.316853 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:08.316910 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:08.320827 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.321694 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.321704 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.321707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.321907 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.322394 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.322400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.322403 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.322848 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:08.323145 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.323800 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.323990 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:14:08.325321 5386 ts_tablet_manager.cc:1918] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:08.325481 5386 log.cc:1196] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/dcc36382a6bb4d2e9c1f9192477fd868
I20260811 12:14:08.325505 4820 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:08.325953 5386 ts_tablet_manager.cc:1939] T dcc36382a6bb4d2e9c1f9192477fd868 P 878fccff21534d469d58862ff21bd5a0: Deleting consensus metadata
I20260811 12:14:08.327467 4441 catalog_manager.cc:4841] TS 878fccff21534d469d58862ff21bd5a0 (127.4.48.194:39127): tablet dcc36382a6bb4d2e9c1f9192477fd868 (table test-table [id=e73bca07f6e34e04af8cec8871ebc3f5]) successfully deleted
I20260811 12:14:08.423182 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4577
I20260811 12:14:08.451033 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4835
Aug 11 12:14:08 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.194@KRBTEST.COM
I20260811 12:14:08.465760 5321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.451821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47510 (local address 127.4.48.194:39127)
0811 12:14:08.452002 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.452008 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.452028 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:14:08.452196 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.452202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.452288 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.452373 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:08.453645 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.454385 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.455629 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.456077 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.456193 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.460448 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:08.460462 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:08.460478 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:08.460521 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:08.463039 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.463716 (+ 677us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.463725 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.463728 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.463860 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.464224 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.464229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.464232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.464609 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:08.464911 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.465229 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.465440 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:14:08.466364 5052 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:08.553427 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4835
I20260811 12:14:08.576921 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5068
Aug 11 12:14:08 dist-test-slave-nj21 krb5kdc[4385](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.195@KRBTEST.COM
I20260811 12:14:08.590746 5323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:08.577639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.4.48.195:32783)
0811 12:14:08.577803 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:08.577808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:08.577843 (+ 35us) server_negotiation.cc:407] Connection header received
0811 12:14:08.577998 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:08.578002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:08.578071 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:08.578149 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:08.579474 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.580233 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.581393 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:08.581754 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:08.581834 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:08.586062 (+ 4228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:08.586076 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:08.586094 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:08.586128 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:08.588592 (+ 2464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.589190 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.589197 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.589199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.589315 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:08.589610 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:08.589612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:08.589614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:08.589942 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:08.590162 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:08.590340 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 12:14:08.590498 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:14:08.591370 5290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:08.677389 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5068
I20260811 12:14:08.700225 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 4408
I20260811 12:14:08.701295 4530 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:08.808041 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 4408
2026-08-11T12:14:08Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4455 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T12:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:08Z Disabled control of system clock
I20260811 12:14:08.900691 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:35601
--webserver_interface=127.4.48.254
--webserver_port=0
--builtin_ntp_servers=127.4.48.212:37713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:35601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:14:09.228304 5398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:09.228588 5398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:09.228722 5398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:09.235895 5398 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:09.235999 5398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:09.236061 5398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:09.236140 5398 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:09.236203 5398 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:09.245326 5398 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:35601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:35601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:09.246590 5398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:09.248428 5398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:09.259796 5405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:09.259893 5407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:09.260027 5409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:09.260741 5398 server_base.cc:1034] running on GCE node
I20260811 12:14:09.261590 5398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:09.262887 5398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:09.264077 5398 hybrid_clock.cc:648] HybridClock initialized: now 1786450449264028 us; error 39 us; skew 500 ppm
I20260811 12:14:09.264387 5398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:09.266428 5398 webserver.cc:460] Webserver started at http://127.4.48.254:40025/ using document root <none> and password file <none>
I20260811 12:14:09.267026 5398 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:09.267108 5398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:09.267369 5398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:09.270133 5398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "74f62b45551540ccbf9c5249ea0572ee"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "7c556a838f8a589753ccdfad1d447b0f"
server_key_iv: "87cbab31ebbc8776ce8ea160b467130a"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.270884 5398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "74f62b45551540ccbf9c5249ea0572ee"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "7c556a838f8a589753ccdfad1d447b0f"
server_key_iv: "87cbab31ebbc8776ce8ea160b467130a"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.276294 5398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:14:09.279439 5419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:09.280668 5398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 12:14:09.280812 5398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "74f62b45551540ccbf9c5249ea0572ee"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "7c556a838f8a589753ccdfad1d447b0f"
server_key_iv: "87cbab31ebbc8776ce8ea160b467130a"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.280974 5398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:09.303774 5398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:09.304515 5398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:09.304751 5398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:09.321871 5398 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:35601
I20260811 12:14:09.321906 5506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:35601 every 8 connection(s)
I20260811 12:14:09.323122 5398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:09.324594 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5398
I20260811 12:14:09.324774 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:09.325369 4291 external_mini_cluster.cc:1428] Setting key 567f40a9a5a072bd79e6f587376e5125
I20260811 12:14:09.327696 5508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:09.338661 5512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:09.327243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.4.48.254:35601)
0811 12:14:09.328211 (+ 968us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:09.328221 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:09.328262 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:09.329873 (+ 1611us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:09.329895 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:09.330325 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:09.330645 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:09.331223 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.332574 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.334424 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.335045 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.335168 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:09.336403 (+ 1235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:09.336411 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:09.336427 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:09.336947 (+ 520us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:09.337027 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:09.337050 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:09.337136 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:09.337372 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 12:14:09.337676 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":359,"threads_started":1}
I20260811 12:14:09.340194 5508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: Bootstrap starting.
I20260811 12:14:09.343256 5508 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:09.344254 5508 log.cc:824] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:09.346957 5508 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: No bootstrap required, opened a new log
I20260811 12:14:09.354041 5508 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } }
I20260811 12:14:09.354372 5508 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:09.354482 5508 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74f62b45551540ccbf9c5249ea0572ee, State: Initialized, Role: FOLLOWER
I20260811 12:14:09.354998 5508 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } }
I20260811 12:14:09.355225 5508 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:09.355347 5508 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:09.355508 5508 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:09.357604 5508 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } }
I20260811 12:14:09.358070 5508 leader_election.cc:302] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74f62b45551540ccbf9c5249ea0572ee; no voters:
I20260811 12:14:09.358614 5508 leader_election.cc:288] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:09.358745 5514 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:09.359930 5514 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [term 1 LEADER]: Becoming Leader. State: Replica: 74f62b45551540ccbf9c5249ea0572ee, State: Running, Role: LEADER
I20260811 12:14:09.360401 5514 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } }
I20260811 12:14:09.360782 5508 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:09.363224 5517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74f62b45551540ccbf9c5249ea0572ee. Latest consensus state: current_term: 1 leader_uuid: "74f62b45551540ccbf9c5249ea0572ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } } }
I20260811 12:14:09.363692 5517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:09.363336 5515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74f62b45551540ccbf9c5249ea0572ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74f62b45551540ccbf9c5249ea0572ee" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 35601 } } }
I20260811 12:14:09.363884 5515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:09.365036 5530 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:09.370005 5530 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:09.379067 5530 catalog_manager.cc:1347] Generated new cluster ID: 7d65ba01fe0341bc926a67b55c5d8178
I20260811 12:14:09.379168 5530 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:09.389145 5530 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:09.390385 5530 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:09.398334 5530 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: Generated new TSK 0
I20260811 12:14:09.399093 5530 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:09.408979 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.193:0
--local_ip_for_outbound_sockets=127.4.48.193
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--builtin_ntp_servers=127.4.48.212:37713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 12:14:09.750550 5547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:09.750813 5547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:09.750931 5547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:09.758009 5547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:09.758106 5547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:09.758255 5547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.193
I20260811 12:14:09.767163 5547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.4.48.193
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.193
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:09.768399 5547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:09.770102 5547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:09.784250 5557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:09.784422 5556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:09.784940 5547 server_base.cc:1034] running on GCE node
W20260811 12:14:09.785432 5559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:09.786254 5547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:09.787405 5547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:09.788654 5547 hybrid_clock.cc:648] HybridClock initialized: now 1786450449788549 us; error 87 us; skew 500 ppm
I20260811 12:14:09.789033 5547 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:09.791807 5547 webserver.cc:460] Webserver started at http://127.4.48.193:41099/ using document root <none> and password file <none>
I20260811 12:14:09.792701 5547 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:09.792826 5547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:09.793206 5547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:09.797191 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data/instance:
uuid: "a93ace0cbdcb4418ac7a535ea887d57f"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "1445399b2a9a069c9755ae7e1cb761d1"
server_key_iv: "0069a76167ac19cf1f939fb64fcf8379"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.798178 5547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance:
uuid: "a93ace0cbdcb4418ac7a535ea887d57f"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "1445399b2a9a069c9755ae7e1cb761d1"
server_key_iv: "0069a76167ac19cf1f939fb64fcf8379"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.804741 5547 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260811 12:14:09.809135 5572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:09.810842 5547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:14:09.811012 5547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal
uuid: "a93ace0cbdcb4418ac7a535ea887d57f"
format_stamp: "Formatted at 2026-08-11 12:14:09 on dist-test-slave-nj21"
server_key: "1445399b2a9a069c9755ae7e1cb761d1"
server_key_iv: "0069a76167ac19cf1f939fb64fcf8379"
server_key_version: "encryptionkey@0"
I20260811 12:14:09.811259 5547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:09.837410 5547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:09.838490 5547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:09.838836 5547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:09.840056 5547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:09.841980 5547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:09.842093 5547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:09.842200 5547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:09.842285 5547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:09.876897 5547 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.193:32957
I20260811 12:14:09.876910 5774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.193:32957 every 8 connection(s)
I20260811 12:14:09.878593 5547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/data/info.pb
I20260811 12:14:09.883919 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5547
I20260811 12:14:09.884096 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-0/wal/instance
I20260811 12:14:09.884675 4291 external_mini_cluster.cc:1428] Setting key 3e6f13b100b02cb6bd7f8454369d4bfb
I20260811 12:14:09.888033 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.194:0
--local_ip_for_outbound_sockets=127.4.48.194
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--builtin_ntp_servers=127.4.48.212:37713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:14:09.893473 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:09.881286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.193:37421 (local address 127.4.48.254:35601)
0811 12:14:09.881688 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:09.881692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:09.883117 (+ 1425us) server_negotiation.cc:407] Connection header received
0811 12:14:09.885535 (+ 2418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:09.885542 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:09.885637 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:09.885743 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:09.886858 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.887944 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.890349 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.890918 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.891038 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:09.892624 (+ 1586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:09.892630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:09.892634 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:09.892656 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:09.892700 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:09.892714 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:09.892795 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:09.893050 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 12:14:09.893144 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":203,"threads_started":1}
I20260811 12:14:09.894804 5779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:09.882007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35601 (local address 127.4.48.193:37421)
0811 12:14:09.882972 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:09.883008 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:09.885258 (+ 2250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:09.885986 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:09.885995 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:09.886079 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:09.886704 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:09.886724 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.888115 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.888122 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:09.890173 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:09.890189 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:09.891148 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:09.891156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:09.891468 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:09.892240 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:09.892244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:09.892298 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:09.892407 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:09.892811 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:09.892822 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:09.892831 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 12:14:09.893017 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 12:14:09.893449 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":318,"threads_started":1}
I20260811 12:14:09.896338 5775 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35601
I20260811 12:14:09.896673 5775 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:09.897588 5775 heartbeater.cc:511] Master 127.4.48.254:35601 requested a full tablet report, sending...
I20260811 12:14:09.900058 5446 ts_manager.cc:194] Registered new tserver with Master: a93ace0cbdcb4418ac7a535ea887d57f (127.4.48.193:32957)
I20260811 12:14:09.902218 5446 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.193:37421
W20260811 12:14:10.223318 5782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:10.223590 5782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:10.223758 5782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:10.230717 5782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:10.230814 5782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:10.230952 5782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.194
I20260811 12:14:10.239660 5782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.4.48.194
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.194
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:10.240957 5782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:10.242780 5782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:10.255124 5796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:10.255228 5792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:10.255162 5793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:10.256012 5782 server_base.cc:1034] running on GCE node
I20260811 12:14:10.256587 5782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:10.257464 5782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:10.258664 5782 hybrid_clock.cc:648] HybridClock initialized: now 1786450450258602 us; error 51 us; skew 500 ppm
I20260811 12:14:10.258958 5782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:10.261103 5782 webserver.cc:460] Webserver started at http://127.4.48.194:34267/ using document root <none> and password file <none>
I20260811 12:14:10.261763 5782 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:10.261864 5782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:10.262171 5782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:10.265065 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data/instance:
uuid: "f2e9afe70eae46168a796437f95492fe"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "6ae484e4345e40dd879cdd53f83c2e7e"
server_key_iv: "ded6f3a79fdf89e4e727a9955b88730a"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.265874 5782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance:
uuid: "f2e9afe70eae46168a796437f95492fe"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "6ae484e4345e40dd879cdd53f83c2e7e"
server_key_iv: "ded6f3a79fdf89e4e727a9955b88730a"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.270690 5782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 12:14:10.274022 5807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.275223 5782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 12:14:10.275377 5782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal
uuid: "f2e9afe70eae46168a796437f95492fe"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "6ae484e4345e40dd879cdd53f83c2e7e"
server_key_iv: "ded6f3a79fdf89e4e727a9955b88730a"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.275574 5782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:10.300204 5782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:10.300977 5782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:10.301259 5782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:10.302210 5782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:10.303602 5782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:10.303714 5782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.303819 5782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:10.303900 5782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.336292 5782 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.194:39501
I20260811 12:14:10.336337 6005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.194:39501 every 8 connection(s)
I20260811 12:14:10.337744 5782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/data/info.pb
I20260811 12:14:10.339414 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 5782
I20260811 12:14:10.339586 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal/instance
I20260811 12:14:10.340139 4291 external_mini_cluster.cc:1428] Setting key 40ceaece1e746af7adb6f779d2160454
I20260811 12:14:10.343153 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.4.48.195:0
--local_ip_for_outbound_sockets=127.4.48.195
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--builtin_ntp_servers=127.4.48.212:37713
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 12:14:10.350296 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.340350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.194:47099 (local address 127.4.48.254:35601)
0811 12:14:10.340635 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.340641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.342235 (+ 1594us) server_negotiation.cc:407] Connection header received
0811 12:14:10.344282 (+ 2047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.344289 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.344366 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.344460 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:10.345414 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.346346 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.347795 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.348291 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.348408 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.349593 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:10.349599 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:10.349602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:10.349621 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:10.349657 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:10.349672 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:10.349756 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.349917 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.350007 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260811 12:14:10.351114 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.340998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35601 (local address 127.4.48.194:47099)
0811 12:14:10.342083 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.342125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.344033 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.344635 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.344642 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.344727 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:10.345259 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.345275 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.346467 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.346470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.347639 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.347651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.348470 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.348475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.348668 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.349290 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:10.349293 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:10.349329 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:10.349405 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:10.349748 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:10.349753 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:10.349758 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.349886 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.350147 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":334,"threads_started":1}
I20260811 12:14:10.352324 6006 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35601
I20260811 12:14:10.352619 6006 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:10.353430 6006 heartbeater.cc:511] Master 127.4.48.254:35601 requested a full tablet report, sending...
I20260811 12:14:10.354748 5446 ts_manager.cc:194] Registered new tserver with Master: f2e9afe70eae46168a796437f95492fe (127.4.48.194:39501)
I20260811 12:14:10.355515 5446 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.194:47099
W20260811 12:14:10.669588 6015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:10.669850 6015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:10.669940 6015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:10.676819 6015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:10.676904 6015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:10.677006 6015 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.4.48.195
I20260811 12:14:10.685729 6015 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37713
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.4.48.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.4.48.195
--webserver_port=0
--tserver_master_addrs=127.4.48.254:35601
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.4.48.195
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:10.686846 6015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:10.688573 6015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:10.700810 6025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:10.700729 6027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:10.700845 6024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:10.701767 6015 server_base.cc:1034] running on GCE node
I20260811 12:14:10.702351 6015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:10.703207 6015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:10.704443 6015 hybrid_clock.cc:648] HybridClock initialized: now 1786450450704385 us; error 43 us; skew 500 ppm
I20260811 12:14:10.704777 6015 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:10.706895 6015 webserver.cc:460] Webserver started at http://127.4.48.195:32813/ using document root <none> and password file <none>
I20260811 12:14:10.707520 6015 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:10.707626 6015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:10.707976 6015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:10.710772 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data/instance:
uuid: "2ab1eb05d3db4206844f6b64d26f58db"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "d0c7b54603579aa1d91a44e601757ccf"
server_key_iv: "e81ad62b795789aabf65cc25b1df47c7"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.711520 6015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance:
uuid: "2ab1eb05d3db4206844f6b64d26f58db"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "d0c7b54603579aa1d91a44e601757ccf"
server_key_iv: "e81ad62b795789aabf65cc25b1df47c7"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.716430 6015 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 12:14:10.719713 6037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.721012 6015 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 12:14:10.721191 6015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal
uuid: "2ab1eb05d3db4206844f6b64d26f58db"
format_stamp: "Formatted at 2026-08-11 12:14:10 on dist-test-slave-nj21"
server_key: "d0c7b54603579aa1d91a44e601757ccf"
server_key_iv: "e81ad62b795789aabf65cc25b1df47c7"
server_key_version: "encryptionkey@0"
I20260811 12:14:10.721414 6015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:10.737884 6015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:10.738710 6015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:10.739014 6015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:10.740024 6015 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 12:14:10.741601 6015 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 12:14:10.741729 6015 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.741847 6015 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 12:14:10.741926 6015 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:10.773370 6015 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.195:36035
I20260811 12:14:10.773361 6233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.195:36035 every 8 connection(s)
I20260811 12:14:10.774770 6015 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/data/info.pb
I20260811 12:14:10.783644 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6015
I20260811 12:14:10.783880 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal/instance
I20260811 12:14:10.784536 4291 external_mini_cluster.cc:1428] Setting key faed9f6c297db08bf3306ecc2b5f56e5
I20260811 12:14:10.787261 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.777234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:52341 (local address 127.4.48.254:35601)
0811 12:14:10.777445 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.777452 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.778913 (+ 1461us) server_negotiation.cc:407] Connection header received
0811 12:14:10.780688 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.780696 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.780776 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.780874 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:10.781804 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.782564 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.784198 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.784933 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.785109 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.786487 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:10.786496 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:10.786501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:10.786523 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:10.786567 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:10.786581 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:10.786668 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.786887 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.786970 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:14:10.788367 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.777793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:35601 (local address 127.4.48.195:52341)
0811 12:14:10.778731 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.778779 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.780447 (+ 1668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.781057 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.781065 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.781143 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:10.781666 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.781682 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.782680 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.782683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.784017 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.784032 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.785195 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.785203 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.785428 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.786154 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 12:14:10.786158 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:10.786207 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 12:14:10.786299 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:10.786678 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:10.786688 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:10.786695 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.786866 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.787225 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":298,"threads_started":1}
I20260811 12:14:10.789587 6234 heartbeater.cc:347] Connected to a master server at 127.4.48.254:35601
I20260811 12:14:10.789824 6234 heartbeater.cc:464] Registering TS with master...
I20260811 12:14:10.790490 6234 heartbeater.cc:511] Master 127.4.48.254:35601 requested a full tablet report, sending...
I20260811 12:14:10.791815 5446 ts_manager.cc:194] Registered new tserver with Master: 2ab1eb05d3db4206844f6b64d26f58db (127.4.48.195:36035)
I20260811 12:14:10.792630 5446 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.4.48.195:52341
I20260811 12:14:10.799396 4291 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 12:14:10.808336 5777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.803331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36852 (local address 127.4.48.254:35601)
0811 12:14:10.803479 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.803484 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.803950 (+ 466us) server_negotiation.cc:407] Connection header received
0811 12:14:10.804074 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.804078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.804146 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.804220 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:10.804621 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.805256 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.806472 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.806854 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.806944 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.807847 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:10.807853 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:10.807856 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:10.807872 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:10.807901 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:10.807912 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:10.807976 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.808060 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.808127 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:14:10.811596 5446 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36852:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 12:14:10.818298 5446 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 12:14:10.849251 6261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.829509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52044)
0811 12:14:10.841307 (+ 11798us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.841324 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.841434 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.842075 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.842082 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.842113 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:10.842367 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.842375 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.844482 (+ 2107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.844488 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.846886 (+ 2398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.846901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.848327 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.848334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.848511 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.848563 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.848700 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.848904 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11666,"thread_start_us":10621,"threads_started":1}
I20260811 12:14:10.850044 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.829664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.4.48.193:32957)
0811 12:14:10.841414 (+ 11750us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.841424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.841458 (+ 34us) server_negotiation.cc:407] Connection header received
0811 12:14:10.841570 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.841578 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.841836 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.842543 (+ 707us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:10.842719 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.844308 (+ 1589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.847147 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.848141 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.848262 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.848525 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.849517 (+ 992us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.849681 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11558,"thread_start_us":11864,"threads_started":1}
I20260811 12:14:10.850060 6262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.829249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39501 (local address 127.0.0.1:54220)
0811 12:14:10.840571 (+ 11322us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.840628 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.840816 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.842776 (+ 1960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.842780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.842807 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:10.843167 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.843180 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.845022 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.845027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.847109 (+ 2082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.847122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.849315 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.849321 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.849527 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.849562 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.849646 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.849768 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11038,"thread_start_us":11762,"threads_started":1}
I20260811 12:14:10.850474 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.829541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.4.48.194:39501)
0811 12:14:10.842219 (+ 12678us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.842228 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.842259 (+ 31us) server_negotiation.cc:407] Connection header received
0811 12:14:10.842354 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.842361 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.842577 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.842768 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:10.843363 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.844840 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.847313 (+ 2473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.848807 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.848940 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.849183 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.850000 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.850164 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12499,"thread_start_us":12371,"threads_started":1}
I20260811 12:14:10.854071 5898 tablet_service.cc:1467] Processing CreateTablet for tablet 85b6309bfc5348f6b6e93a871d206018 (DEFAULT_TABLE table=test-table [id=b4dbbc010a18458892fb11c1d0406f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:10.854065 5658 tablet_service.cc:1467] Processing CreateTablet for tablet 85b6309bfc5348f6b6e93a871d206018 (DEFAULT_TABLE table=test-table [id=b4dbbc010a18458892fb11c1d0406f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:10.855494 5658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b6309bfc5348f6b6e93a871d206018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:10.855490 5898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b6309bfc5348f6b6e93a871d206018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:10.856025 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.843174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.195:36035 (local address 127.0.0.1:60176)
0811 12:14:10.848925 (+ 5751us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.848941 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.849054 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.850428 (+ 1374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.850433 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.850459 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:10.850754 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.850766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.852522 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.852527 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.853876 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.853891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.855162 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.855169 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.855345 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.855375 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.855463 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.855725 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5625,"thread_start_us":5484,"threads_started":1}
I20260811 12:14:10.856846 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.841789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60176 (local address 127.4.48.195:36035)
0811 12:14:10.846482 (+ 4693us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.846495 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.849170 (+ 2675us) server_negotiation.cc:407] Connection header received
0811 12:14:10.849271 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.849279 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.849800 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.849977 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:10.851063 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.852349 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.854059 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.854981 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.855099 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.856360 (+ 1261us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.856451 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.856585 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4562,"thread_start_us":4416,"threads_started":1}
I20260811 12:14:10.859953 6121 tablet_service.cc:1467] Processing CreateTablet for tablet 85b6309bfc5348f6b6e93a871d206018 (DEFAULT_TABLE table=test-table [id=b4dbbc010a18458892fb11c1d0406f7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 12:14:10.861307 6121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85b6309bfc5348f6b6e93a871d206018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:10.870002 6273 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Bootstrap starting.
I20260811 12:14:10.871059 6272 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Bootstrap starting.
I20260811 12:14:10.872634 6275 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Bootstrap starting.
I20260811 12:14:10.874408 6273 tablet_bootstrap.cc:654] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:10.874949 6272 tablet_bootstrap.cc:654] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:10.875388 6273 log.cc:824] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:10.875918 6275 tablet_bootstrap.cc:654] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:10.876186 6272 log.cc:824] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:10.877164 6275 log.cc:824] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:10.878595 6273 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: No bootstrap required, opened a new log
I20260811 12:14:10.878966 6273 ts_tablet_manager.cc:1397] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 12:14:10.883236 6275 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: No bootstrap required, opened a new log
I20260811 12:14:10.883519 6275 ts_tablet_manager.cc:1397] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 12:14:10.889484 6273 raft_consensus.cc:348] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.890110 6273 raft_consensus.cc:374] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:10.890159 6272 tablet_bootstrap.cc:492] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: No bootstrap required, opened a new log
I20260811 12:14:10.890220 6273 raft_consensus.cc:729] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2e9afe70eae46168a796437f95492fe, State: Initialized, Role: FOLLOWER
I20260811 12:14:10.890394 6272 ts_tablet_manager.cc:1397] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Time spent bootstrapping tablet: real 0.020s user 0.009s sys 0.006s
I20260811 12:14:10.890763 6273 consensus_queue.cc:260] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.891806 6006 heartbeater.cc:503] Master 127.4.48.254:35601 was elected leader, sending a full tablet report...
I20260811 12:14:10.892468 6273 ts_tablet_manager.cc:1428] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260811 12:14:10.894373 6275 raft_consensus.cc:348] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.894780 6275 raft_consensus.cc:374] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:10.895623 6275 raft_consensus.cc:729] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab1eb05d3db4206844f6b64d26f58db, State: Initialized, Role: FOLLOWER
I20260811 12:14:10.896224 6275 consensus_queue.cc:260] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.897578 6234 heartbeater.cc:503] Master 127.4.48.254:35601 was elected leader, sending a full tablet report...
I20260811 12:14:10.897922 6275 ts_tablet_manager.cc:1428] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 12:14:10.901185 6272 raft_consensus.cc:348] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.901490 6272 raft_consensus.cc:374] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:10.901587 6272 raft_consensus.cc:729] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a93ace0cbdcb4418ac7a535ea887d57f, State: Initialized, Role: FOLLOWER
I20260811 12:14:10.902127 6272 consensus_queue.cc:260] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.903021 5775 heartbeater.cc:503] Master 127.4.48.254:35601 was elected leader, sending a full tablet report...
I20260811 12:14:10.903357 6272 ts_tablet_manager.cc:1428] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Time spent starting tablet: real 0.013s user 0.011s sys 0.000s
I20260811 12:14:10.972405 6284 raft_consensus.cc:482] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 12:14:10.972675 6284 raft_consensus.cc:504] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.974285 6284 leader_election.cc:288] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2e9afe70eae46168a796437f95492fe (127.4.48.194:39501), a93ace0cbdcb4418ac7a535ea887d57f (127.4.48.193:32957)
I20260811 12:14:10.979621 6240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.974536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.4.48.195:60535)
0811 12:14:10.974788 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.974813 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.974933 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.975325 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.975329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.975359 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:10.975523 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.975530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.976516 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.976521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.977845 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.977859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.978983 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.978987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.979134 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.979169 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.979257 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.979404 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135}
I20260811 12:14:10.979736 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.974692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:60535 (local address 127.4.48.193:32957)
0811 12:14:10.974879 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.974885 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.974907 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:14:10.975079 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.975085 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.975177 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.975279 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:10.975700 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.976387 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.978010 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.978832 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.978997 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.979069 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.979319 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.979448 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:14:10.980633 6286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.974618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.194:39501 (local address 127.4.48.195:34069)
0811 12:14:10.975438 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:10.975453 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:10.975574 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:10.975892 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:10.975897 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:10.975919 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 12:14:10.976235 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.976244 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.977205 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.977209 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.978498 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:10.978509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.980027 (+ 1518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.980032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:10.980182 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.980213 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 12:14:10.980309 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 12:14:10.980423 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":451,"threads_started":1}
I20260811 12:14:10.980701 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:10.974692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.4.48.195:34069 (local address 127.4.48.194:39501)
0811 12:14:10.974893 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:10.974899 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:10.975553 (+ 654us) server_negotiation.cc:407] Connection header received
0811 12:14:10.975728 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:10.975733 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:10.975792 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:10.975863 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 12:14:10.976393 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.977059 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.978688 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:10.979810 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:10.979968 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:10.980035 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:10.980363 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 12:14:10.980484 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:14:10.980930 5700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b6309bfc5348f6b6e93a871d206018" candidate_uuid: "2ab1eb05d3db4206844f6b64d26f58db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" is_pre_election: true
I20260811 12:14:10.981432 5700 raft_consensus.cc:2454] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab1eb05d3db4206844f6b64d26f58db in term 0.
I20260811 12:14:10.981585 5929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b6309bfc5348f6b6e93a871d206018" candidate_uuid: "2ab1eb05d3db4206844f6b64d26f58db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e9afe70eae46168a796437f95492fe" is_pre_election: true
I20260811 12:14:10.982025 6048 leader_election.cc:302] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1eb05d3db4206844f6b64d26f58db, a93ace0cbdcb4418ac7a535ea887d57f; no voters:
I20260811 12:14:10.982070 5929 raft_consensus.cc:2454] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ab1eb05d3db4206844f6b64d26f58db in term 0.
I20260811 12:14:10.982372 6284 raft_consensus.cc:2781] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 12:14:10.982547 6284 raft_consensus.cc:482] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 12:14:10.982653 6284 raft_consensus.cc:3037] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:10.984854 6284 raft_consensus.cc:504] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.985390 6284 leader_election.cc:288] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [CANDIDATE]: Term 1 election: Requested vote from peers f2e9afe70eae46168a796437f95492fe (127.4.48.194:39501), a93ace0cbdcb4418ac7a535ea887d57f (127.4.48.193:32957)
I20260811 12:14:10.985775 5929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b6309bfc5348f6b6e93a871d206018" candidate_uuid: "2ab1eb05d3db4206844f6b64d26f58db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2e9afe70eae46168a796437f95492fe"
I20260811 12:14:10.985814 5700 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85b6309bfc5348f6b6e93a871d206018" candidate_uuid: "2ab1eb05d3db4206844f6b64d26f58db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a93ace0cbdcb4418ac7a535ea887d57f"
I20260811 12:14:10.985939 5929 raft_consensus.cc:3037] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:10.985981 5700 raft_consensus.cc:3037] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:10.988132 5929 raft_consensus.cc:2454] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab1eb05d3db4206844f6b64d26f58db in term 1.
I20260811 12:14:10.988164 5700 raft_consensus.cc:2454] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ab1eb05d3db4206844f6b64d26f58db in term 1.
I20260811 12:14:10.988485 6045 leader_election.cc:302] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ab1eb05d3db4206844f6b64d26f58db, f2e9afe70eae46168a796437f95492fe; no voters:
I20260811 12:14:10.988790 6284 raft_consensus.cc:2781] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:10.989216 6284 raft_consensus.cc:686] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 1 LEADER]: Becoming Leader. State: Replica: 2ab1eb05d3db4206844f6b64d26f58db, State: Running, Role: LEADER
I20260811 12:14:10.989660 6284 consensus_queue.cc:237] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } }
I20260811 12:14:10.994112 5446 catalog_manager.cc:5495] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ab1eb05d3db4206844f6b64d26f58db (127.4.48.195). New cstate: current_term: 1 leader_uuid: "2ab1eb05d3db4206844f6b64d26f58db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ab1eb05d3db4206844f6b64d26f58db" member_type: VOTER last_known_addr { host: "127.4.48.195" port: 36035 } health_report { overall_health: HEALTHY } } }
I20260811 12:14:11.008433 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.003772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60184 (local address 127.4.48.195:36035)
0811 12:14:11.003968 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:11.003974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:11.004099 (+ 125us) server_negotiation.cc:407] Connection header received
0811 12:14:11.004265 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:11.004269 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:11.004340 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:11.004448 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:11.004714 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.005406 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.006647 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.007029 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.007111 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:11.007478 (+ 367us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:11.007983 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:11.008094 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:11.008150 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 12:14:11.008209 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 12:14:11.015951 5700 raft_consensus.cc:1261] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 1 FOLLOWER]: Refusing update from remote peer 2ab1eb05d3db4206844f6b64d26f58db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:11.015947 5929 raft_consensus.cc:1261] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 1 FOLLOWER]: Refusing update from remote peer 2ab1eb05d3db4206844f6b64d26f58db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:11.016901 6287 consensus_queue.cc:1035] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [LEADER]: Connected to new peer: Peer: permanent_uuid: "a93ace0cbdcb4418ac7a535ea887d57f" member_type: VOTER last_known_addr { host: "127.4.48.193" port: 32957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:11.017225 6284 consensus_queue.cc:1035] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2e9afe70eae46168a796437f95492fe" member_type: VOTER last_known_addr { host: "127.4.48.194" port: 39501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 12:14:11.026415 6238 tablet.cc:2369] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:11.041507 5445 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:11.041838 5445 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36852:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 12:14:11.045761 5445 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74f62b45551540ccbf9c5249ea0572ee: Sending DeleteTablet for 3 replicas of tablet 85b6309bfc5348f6b6e93a871d206018
I20260811 12:14:11.046914 5658 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b6309bfc5348f6b6e93a871d206018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:11 UTC) from {username='slave'} at 127.0.0.1:52044
I20260811 12:14:11.046914 5898 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b6309bfc5348f6b6e93a871d206018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:11 UTC) from {username='slave'} at 127.0.0.1:54220
I20260811 12:14:11.047037 6121 tablet_service.cc:1514] Processing DeleteTablet for tablet 85b6309bfc5348f6b6e93a871d206018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 12:14:11 UTC) from {username='slave'} at 127.0.0.1:60176
I20260811 12:14:11.052807 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5547
I20260811 12:14:11.059587 6307 tablet_replica.cc:330] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: stopping tablet replica
I20260811 12:14:11.059311 6312 tablet_replica.cc:330] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: stopping tablet replica
I20260811 12:14:11.060235 6307 raft_consensus.cc:2229] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:11.060616 6312 raft_consensus.cc:2229] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 1 LEADER]: Raft consensus shutting down.
I20260811 12:14:11.060973 6307 raft_consensus.cc:2258] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:11.061275 6312 raft_consensus.cc:2258] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:11.063557 6307 ts_tablet_manager.cc:1905] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:11.064149 6310 tablet_replica.cc:330] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: stopping tablet replica
I20260811 12:14:11.064776 6312 ts_tablet_manager.cc:1905] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:11.065696 6310 raft_consensus.cc:2229] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 12:14:11.072023 6307 ts_tablet_manager.cc:1918] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:11.072181 6307 log.cc:1196] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-1/wal/wals/85b6309bfc5348f6b6e93a871d206018
I20260811 12:14:11.072453 6312 ts_tablet_manager.cc:1918] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 12:14:11.072614 6312 log.cc:1196] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Deleting WAL directory at /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786450132843697-4291-0/minicluster-data/ts-2/wal/wals/85b6309bfc5348f6b6e93a871d206018
I20260811 12:14:11.072737 6307 ts_tablet_manager.cc:1939] T 85b6309bfc5348f6b6e93a871d206018 P f2e9afe70eae46168a796437f95492fe: Deleting consensus metadata
I20260811 12:14:11.073124 6312 ts_tablet_manager.cc:1939] T 85b6309bfc5348f6b6e93a871d206018 P 2ab1eb05d3db4206844f6b64d26f58db: Deleting consensus metadata
I20260811 12:14:11.074515 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.054238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.4.48.193:32957)
0811 12:14:11.054433 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:11.054439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:11.065832 (+ 11393us) server_negotiation.cc:407] Connection header received
0811 12:14:11.066335 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:11.066341 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:11.066468 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:11.066549 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:11.067223 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.068423 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.070370 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.071019 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.071117 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:11.072755 (+ 1638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:11.072768 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:11.072785 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:11.073520 (+ 735us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:11.073581 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:11.073605 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:11.073699 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:11.074018 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 12:14:11.074134 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:14:11.074786 6310 raft_consensus.cc:2258] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 12:14:11.074854 5423 catalog_manager.cc:4841] TS f2e9afe70eae46168a796437f95492fe (127.4.48.194:39501): tablet 85b6309bfc5348f6b6e93a871d206018 (table test-table [id=b4dbbc010a18458892fb11c1d0406f7b]) successfully deleted
I20260811 12:14:11.075443 5423 catalog_manager.cc:4841] TS 2ab1eb05d3db4206844f6b64d26f58db (127.4.48.195:36035): tablet 85b6309bfc5348f6b6e93a871d206018 (table test-table [id=b4dbbc010a18458892fb11c1d0406f7b]) successfully deleted
I20260811 12:14:11.076607 6310 ts_tablet_manager.cc:1905] T 85b6309bfc5348f6b6e93a871d206018 P a93ace0cbdcb4418ac7a535ea887d57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 12:14:11.076642 5769 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:11.167835 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5547
I20260811 12:14:11.197232 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5782
W20260811 12:14:11.197268 5424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.4.48.193:32957 (error 108)
I20260811 12:14:11.199856 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.199327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52070)
0811 12:14:11.199541 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.199705 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 12:14:11.200101 5424 catalog_manager.cc:4584] TS a93ace0cbdcb4418ac7a535ea887d57f (127.4.48.193:32957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85b6309bfc5348f6b6e93a871d206018: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
I20260811 12:14:11.205749 6263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.198243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54232 (local address 127.4.48.194:39501)
0811 12:14:11.198870 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:11.198876 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:11.198898 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:14:11.198968 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:11.198973 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:11.199084 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:11.199206 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:11.199728 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.200821 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.202132 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.202809 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.202955 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:11.204108 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:11.204122 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:11.204141 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:11.204944 (+ 803us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:11.205009 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:11.205038 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:11.205173 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:11.205229 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 12:14:11.205351 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":488}
I20260811 12:14:11.206450 6000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:11.253793 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.253203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52084)
0811 12:14:11.253461 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.253576 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 12:14:11.254937 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.254554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52088)
0811 12:14:11.254706 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.254796 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 12:14:11.293524 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5782
I20260811 12:14:11.315841 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.315233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52098)
0811 12:14:11.315473 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.315590 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 12:14:11.317128 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.316685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52112)
0811 12:14:11.316866 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.316973 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:14:11.321745 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6015
I20260811 12:14:11.350983 6265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.322662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60200 (local address 127.4.48.195:36035)
0811 12:14:11.322856 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:11.322860 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:11.322875 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:14:11.322928 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:11.322932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:11.323006 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:11.323089 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:11.323528 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.324368 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.325698 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:11.326182 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:11.326292 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:11.349491 (+ 23199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:11.349501 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:11.349523 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:11.350137 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:11.350192 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:11.350219 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:11.350358 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:11.350552 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 12:14:11.350649 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:14:11.351783 6227 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:11.397804 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.397338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52116)
0811 12:14:11.397547 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.397645 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:14:11.398937 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.398490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52130)
0811 12:14:11.398708 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.398822 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 12:14:11.470355 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6015
I20260811 12:14:11.492796 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 5398
I20260811 12:14:11.493887 5498 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:11.598325 6270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:11.597615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.193:32957 (local address 127.0.0.1:52144)
0811 12:14:11.597956 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:11.598089 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.193:32957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260811 12:14:11.598513 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 5398
2026-08-11T12:14:11Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2789 ms)
[----------] 6 tests from EncryptionPolicyTest (21900 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:11 dist-test-slave-nj21 krb5kdc[6327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:11 dist-test-slave-nj21 krb5kdc[6327](info): set up 2 sockets
Aug 11 12:14:11 dist-test-slave-nj21 krb5kdc[6327](info): commencing operation
krb5kdc: starting...
W20260811 12:14:12.837009 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.005s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:13.114925 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40545
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40545
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:13.445312 6355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:13.445659 6355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:13.445767 6355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:13.452800 6355 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:13.452895 6355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:13.452947 6355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:13.453006 6355 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:13.453047 6355 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:13.462359 6355 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40545
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40545
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:13.463507 6355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:13.464769 6355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:13.475606 6361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:13.475720 6362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:13.475766 6367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:13.476760 6355 server_base.cc:1034] running on GCE node
I20260811 12:14:13.477473 6355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:13.478772 6355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:13.479969 6355 hybrid_clock.cc:648] HybridClock initialized: now 1786450453479917 us; error 37 us; skew 500 ppm
I20260811 12:14:13.480288 6355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:13.483161 6355 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:13.484584 6355 webserver.cc:460] Webserver started at http://127.0.0.1:41009/ using document root <none> and password file <none>
I20260811 12:14:13.485208 6355 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:13.485304 6355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:13.485562 6355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:13.488425 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "f899ef3b701241a79873912d7ae7067a"
format_stamp: "Formatted at 2026-08-11 12:14:13 on dist-test-slave-nj21"
server_key: "5504d1582c264e535d56c5501aea049a"
server_key_iv: "eceedd32912d0e403c7df199a259779d"
server_key_version: "encryptionkey@0"
I20260811 12:14:13.489132 6355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "f899ef3b701241a79873912d7ae7067a"
format_stamp: "Formatted at 2026-08-11 12:14:13 on dist-test-slave-nj21"
server_key: "5504d1582c264e535d56c5501aea049a"
server_key_iv: "eceedd32912d0e403c7df199a259779d"
server_key_version: "encryptionkey@0"
I20260811 12:14:13.494012 6355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:14:13.497234 6379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:13.498432 6355 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:14:13.498580 6355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "f899ef3b701241a79873912d7ae7067a"
format_stamp: "Formatted at 2026-08-11 12:14:13 on dist-test-slave-nj21"
server_key: "5504d1582c264e535d56c5501aea049a"
server_key_iv: "eceedd32912d0e403c7df199a259779d"
server_key_version: "encryptionkey@0"
I20260811 12:14:13.498751 6355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:13.527129 6355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:13.530975 6355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:13.531239 6355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:13.548552 6355 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40545
I20260811 12:14:13.548573 6463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40545 every 8 connection(s)
I20260811 12:14:13.549926 6355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:13.550925 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6355
I20260811 12:14:13.551136 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:13.551839 4291 external_mini_cluster.cc:1428] Setting key 7f2efb72060c6479777cef7a30c02eb0
I20260811 12:14:13.554014 6464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:13.566447 6464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a: Bootstrap starting.
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:14:13.569726 6464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:13.570716 6464 log.cc:824] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:13.573285 6464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a: No bootstrap required, opened a new log
I20260811 12:14:13.575625 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:13.553741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54751 (local address 127.0.0.1:40545)
0811 12:14:13.554566 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:13.554581 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:13.554629 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:14:13.556795 (+ 2166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:13.556822 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:13.557304 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:13.557725 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:13.558973 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:13.560990 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:13.562211 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:13.563198 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:13.563379 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:13.567979 (+ 4600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:13.567990 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:13.568003 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:13.568051 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:13.571205 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:13.572130 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:13.572137 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:13.572143 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:13.572328 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:13.572848 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:13.572852 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:13.572854 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:13.573387 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:13.573717 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:13.574258 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 12:14:13.574567 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":323,"threads_started":1}
I20260811 12:14:13.579816 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:13.581451 6464 raft_consensus.cc:348] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260811 12:14:13.581768 6464 raft_consensus.cc:374] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:13.581852 6464 raft_consensus.cc:729] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f899ef3b701241a79873912d7ae7067a, State: Initialized, Role: FOLLOWER
I20260811 12:14:13.582394 6464 consensus_queue.cc:260] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260811 12:14:13.582614 6464 raft_consensus.cc:388] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:13.582744 6464 raft_consensus.cc:482] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:13.582895 6464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:13.585114 6464 raft_consensus.cc:504] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260811 12:14:13.585673 6464 leader_election.cc:302] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f899ef3b701241a79873912d7ae7067a; no voters:
I20260811 12:14:13.586292 6464 leader_election.cc:288] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:13.586406 6479 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:13.587824 6479 raft_consensus.cc:686] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [term 1 LEADER]: Becoming Leader. State: Replica: f899ef3b701241a79873912d7ae7067a, State: Running, Role: LEADER
I20260811 12:14:13.588426 6464 sys_catalog.cc:564] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:13.588502 6479 consensus_queue.cc:237] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } }
I20260811 12:14:13.596539 6481 sys_catalog.cc:455] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f899ef3b701241a79873912d7ae7067a. Latest consensus state: current_term: 1 leader_uuid: "f899ef3b701241a79873912d7ae7067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } } }
I20260811 12:14:13.596946 6481 sys_catalog.cc:458] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:13.597468 6480 sys_catalog.cc:455] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f899ef3b701241a79873912d7ae7067a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f899ef3b701241a79873912d7ae7067a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40545 } } }
I20260811 12:14:13.597726 6480 sys_catalog.cc:458] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:13.599164 6497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:13.603156 6467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:13.584742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45063 (local address 127.0.0.1:40545)
0811 12:14:13.584937 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:13.584944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:13.585364 (+ 420us) server_negotiation.cc:407] Connection header received
0811 12:14:13.585547 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:13.585551 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:13.585623 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:13.585730 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:13.587556 (+ 1826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:13.588884 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:13.590681 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:13.591615 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:13.591851 (+ 236us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:13.595376 (+ 3525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:13.595383 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:13.595387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:13.595431 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:13.599177 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:13.600029 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:13.600035 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:13.600038 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:13.600302 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:13.600788 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:13.600793 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:13.600796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:13.601193 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:13.602421 (+ 1228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:13.602617 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:14:13.602797 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:14:13.605773 6497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:13.614239 6497 catalog_manager.cc:1347] Generated new cluster ID: 442b34665e314a1e8af8d7dca95fb8f2
I20260811 12:14:13.614333 6497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:13.627621 6497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:13.629333 6497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:13.646559 6497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f899ef3b701241a79873912d7ae7067a: Generated new TSK 0
I20260811 12:14:13.647311 6497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:13.662066 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6355
I20260811 12:14:13.663184 6455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:13.759473 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6355
W20260811 12:14:13.785286 6323 connection.cc:535] client connection to 127.0.0.1:40545 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40545): Connection reset by peer (error 104)
2026-08-11T12:14:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2153 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6516](info): set up 2 sockets
Aug 11 12:14:13 dist-test-slave-nj21 krb5kdc[6516](info): commencing operation
krb5kdc: starting...
W20260811 12:14:15.014192 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.210s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:15 dist-test-slave-nj21 krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:15.289426 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:15.619123 6539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:15.619415 6539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:15.619557 6539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:15.626542 6539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:15.626657 6539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:15.626739 6539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:15.626832 6539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:15.626902 6539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:15.636428 6539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:15.637670 6539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:15.638965 6539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:15.650099 6545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:15.650144 6546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:15.650167 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:15.651264 6539 server_base.cc:1034] running on GCE node
I20260811 12:14:15.652053 6539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:15.653366 6539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:15.654568 6539 hybrid_clock.cc:648] HybridClock initialized: now 1786450455654524 us; error 36 us; skew 500 ppm
I20260811 12:14:15.654906 6539 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:15 dist-test-slave-nj21 krb5kdc[6516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450455, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:15.657961 6539 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:15.659381 6539 webserver.cc:460] Webserver started at http://127.0.0.1:39503/ using document root <none> and password file <none>
I20260811 12:14:15.660097 6539 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:15.660214 6539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:15.660506 6539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:15.663383 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "cc14cf496b5345ad932d89e892e766c6"
format_stamp: "Formatted at 2026-08-11 12:14:15 on dist-test-slave-nj21"
server_key: "eb31ecedb69d520f853a24fab8787a1b"
server_key_iv: "e87b73a77a5a54a427e179fb779df6be"
server_key_version: "encryptionkey@0"
I20260811 12:14:15.664144 6539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "cc14cf496b5345ad932d89e892e766c6"
format_stamp: "Formatted at 2026-08-11 12:14:15 on dist-test-slave-nj21"
server_key: "eb31ecedb69d520f853a24fab8787a1b"
server_key_iv: "e87b73a77a5a54a427e179fb779df6be"
server_key_version: "encryptionkey@0"
I20260811 12:14:15.668954 6539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:14:15.672165 6561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:15.673375 6539 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 12:14:15.673530 6539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "cc14cf496b5345ad932d89e892e766c6"
format_stamp: "Formatted at 2026-08-11 12:14:15 on dist-test-slave-nj21"
server_key: "eb31ecedb69d520f853a24fab8787a1b"
server_key_iv: "e87b73a77a5a54a427e179fb779df6be"
server_key_version: "encryptionkey@0"
I20260811 12:14:15.673707 6539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:15.689002 6539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:15.692682 6539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:15.692934 6539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:15.709864 6539 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40269
I20260811 12:14:15.709896 6641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40269 every 8 connection(s)
I20260811 12:14:15.711135 6539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:15.714711 6642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:15.714991 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6539
I20260811 12:14:15.715178 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:15.715754 4291 external_mini_cluster.cc:1428] Setting key c11bc6c79cb77825af100ed092525031
I20260811 12:14:15.727771 6642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6: Bootstrap starting.
I20260811 12:14:15.728180 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:15.717540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35059 (local address 127.0.0.1:40269)
0811 12:14:15.718319 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:15.718330 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:15.718390 (+ 60us) server_negotiation.cc:407] Connection header received
0811 12:14:15.719944 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:15.719969 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:15.720334 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:15.720690 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:15.721224 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:15.722485 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:15.723808 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:15.724503 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:15.724682 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:15.725978 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:15.725986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:15.725999 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:15.726530 (+ 531us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:15.726601 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:15.726626 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:15.726770 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:15.727046 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 12:14:15.727192 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":347,"threads_started":1}
I20260811 12:14:15.731204 6642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:15.731767 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:15.732422 6642 log.cc:824] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:15.735496 6642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6: No bootstrap required, opened a new log
I20260811 12:14:15.743193 6645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:15.736506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56271 (local address 127.0.0.1:40269)
0811 12:14:15.736673 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:15.736678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:15.737025 (+ 347us) server_negotiation.cc:407] Connection header received
0811 12:14:15.737227 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:15.737231 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:15.737287 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:15.737381 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:15.737998 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:15.738759 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:15.740732 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:15.741190 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:15.741281 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:15.742448 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:15.742453 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:15.742456 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:15.742470 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:15.742525 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:15.742539 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:15.742622 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:15.742839 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:14:15.742882 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:14:15.745486 6642 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260811 12:14:15.745855 6642 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:15.745958 6642 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc14cf496b5345ad932d89e892e766c6, State: Initialized, Role: FOLLOWER
I20260811 12:14:15.746615 6642 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260811 12:14:15.746912 6642 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:15.747057 6642 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:15.747231 6642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:15.749599 6642 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260811 12:14:15.750108 6642 leader_election.cc:302] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc14cf496b5345ad932d89e892e766c6; no voters:
I20260811 12:14:15.750674 6642 leader_election.cc:288] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:15.750771 6658 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:15.751554 6658 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [term 1 LEADER]: Becoming Leader. State: Replica: cc14cf496b5345ad932d89e892e766c6, State: Running, Role: LEADER
I20260811 12:14:15.752116 6658 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } }
I20260811 12:14:15.753118 6642 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:15.754797 6659 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc14cf496b5345ad932d89e892e766c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } } }
I20260811 12:14:15.754900 6660 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc14cf496b5345ad932d89e892e766c6. Latest consensus state: current_term: 1 leader_uuid: "cc14cf496b5345ad932d89e892e766c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc14cf496b5345ad932d89e892e766c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40269 } } }
I20260811 12:14:15.755251 6660 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:15.755251 6659 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:15.756801 6667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:15.761389 6667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:15.771166 6667 catalog_manager.cc:1347] Generated new cluster ID: b9e3dd052cc74d038e5289196a773ede
I20260811 12:14:15.771266 6667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:15.780819 6667 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:15.781949 6667 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:15.787652 6667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc14cf496b5345ad932d89e892e766c6: Generated new TSK 0
I20260811 12:14:15.788362 6667 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:15.802532 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6539
W20260811 12:14:15.803785 6634 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35059
W20260811 12:14:15.804647 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6539: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:15.804858 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6539
W20260811 12:14:15.829133 6512 connection.cc:535] client connection to 127.0.0.1:40269 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40269): Connection reset by peer (error 104)
2026-08-11T12:14:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2044 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:15 dist-test-slave-nj21 krb5kdc[6690](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:15 dist-test-slave-nj21 krb5kdc[6690](info): set up 2 sockets
Aug 11 12:14:15 dist-test-slave-nj21 krb5kdc[6690](info): commencing operation
krb5kdc: starting...
W20260811 12:14:17.036245 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:17 dist-test-slave-nj21 krb5kdc[6690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:17.318571 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:17.647261 6712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:17.647548 6712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:17.647708 6712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:17.654788 6712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:17.654906 6712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:17.654982 6712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:17.655062 6712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:17.655122 6712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:17.664655 6712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:17.665895 6712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:17.667171 6712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:17.678663 6721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:17.678753 6720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:17.680050 6726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:17.680418 6712 server_base.cc:1034] running on GCE node
I20260811 12:14:17.681304 6712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:17.683204 6712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:17.684435 6712 hybrid_clock.cc:648] HybridClock initialized: now 1786450457684331 us; error 94 us; skew 500 ppm
I20260811 12:14:17.684782 6712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:17 dist-test-slave-nj21 krb5kdc[6690](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450457, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:17.692478 6712 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:17.693922 6712 webserver.cc:460] Webserver started at http://127.0.0.1:35687/ using document root <none> and password file <none>
I20260811 12:14:17.694621 6712 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:17.694741 6712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:17.695039 6712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:17.698035 6712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b"
format_stamp: "Formatted at 2026-08-11 12:14:17 on dist-test-slave-nj21"
server_key: "af4bf8c813bf14267363ccc209e1af69"
server_key_iv: "caff847fa6d752f5702b95ff74ba1e67"
server_key_version: "encryptionkey@0"
I20260811 12:14:17.698783 6712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b"
format_stamp: "Formatted at 2026-08-11 12:14:17 on dist-test-slave-nj21"
server_key: "af4bf8c813bf14267363ccc209e1af69"
server_key_iv: "caff847fa6d752f5702b95ff74ba1e67"
server_key_version: "encryptionkey@0"
I20260811 12:14:17.704041 6712 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:14:17.707393 6740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:17.708702 6712 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:17.708865 6712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b"
format_stamp: "Formatted at 2026-08-11 12:14:17 on dist-test-slave-nj21"
server_key: "af4bf8c813bf14267363ccc209e1af69"
server_key_iv: "caff847fa6d752f5702b95ff74ba1e67"
server_key_version: "encryptionkey@0"
I20260811 12:14:17.709046 6712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:17.714473 6712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:17.718505 6712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:17.718904 6712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:17.737593 6712 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43883
I20260811 12:14:17.737630 6835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43883 every 8 connection(s)
I20260811 12:14:17.739024 6712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:17.742632 6836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:17.744302 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6712
I20260811 12:14:17.744462 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:17.744928 4291 external_mini_cluster.cc:1428] Setting key 8561d2e239953e0c5949e6e823cb8543
I20260811 12:14:17.752753 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:17.746626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39583 (local address 127.0.0.1:43883)
0811 12:14:17.747545 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:17.747561 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:17.747609 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:14:17.749394 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:17.749419 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:17.749747 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:17.750063 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:17.751119 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:17.751126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:17.751144 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:17.751159 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:17.751247 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:17.751276 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:17.751470 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:17.751581 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 12:14:17.751819 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":406,"threads_started":1}
I20260811 12:14:17.753877 6836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b: Bootstrap starting.
I20260811 12:14:17.756455 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:17.757126 6836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:17.758296 6836 log.cc:824] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:17.761468 6836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b: No bootstrap required, opened a new log
I20260811 12:14:17.763835 6840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:17.761545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37009 (local address 127.0.0.1:43883)
0811 12:14:17.761673 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:17.761678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:17.762062 (+ 384us) server_negotiation.cc:407] Connection header received
0811 12:14:17.762217 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:17.762220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:17.762263 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:17.762344 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:17.763223 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:17.763227 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:17.763230 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:17.763247 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:17.763285 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:17.763299 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:17.763391 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:17.763490 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 12:14:17.763537 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 12:14:17.769148 6836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } }
I20260811 12:14:17.769496 6836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:17.769593 6836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e6a711e4f3b4b6cb5522bc2b0c1156b, State: Initialized, Role: FOLLOWER
I20260811 12:14:17.770109 6836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } }
I20260811 12:14:17.770316 6836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:17.770617 6836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:17.770857 6836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:17.773083 6836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } }
I20260811 12:14:17.773550 6836 leader_election.cc:302] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e6a711e4f3b4b6cb5522bc2b0c1156b; no voters:
I20260811 12:14:17.774080 6836 leader_election.cc:288] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:17.774190 6852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:17.774806 6852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [term 1 LEADER]: Becoming Leader. State: Replica: 6e6a711e4f3b4b6cb5522bc2b0c1156b, State: Running, Role: LEADER
I20260811 12:14:17.775382 6852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } }
I20260811 12:14:17.776028 6836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:17.778151 6858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e6a711e4f3b4b6cb5522bc2b0c1156b. Latest consensus state: current_term: 1 leader_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } } }
I20260811 12:14:17.778602 6858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:17.778112 6856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e6a711e4f3b4b6cb5522bc2b0c1156b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43883 } } }
I20260811 12:14:17.779191 6856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:17.780297 6865 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:17.784765 6865 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:17.792761 6865 catalog_manager.cc:1347] Generated new cluster ID: a30db7ee221f47bc844939a178579003
I20260811 12:14:17.792872 6865 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:17.836474 6865 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:17.836890 6865 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:17.847178 6865 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e6a711e4f3b4b6cb5522bc2b0c1156b: Generated new TSK 0
I20260811 12:14:17.847985 6865 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:17.864660 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6712
W20260811 12:14:17.865837 6828 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39583
W20260811 12:14:17.866549 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6712: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:17.866688 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6712
2026-08-11T12:14:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2066 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:17 dist-test-slave-nj21 krb5kdc[6890](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:17 dist-test-slave-nj21 krb5kdc[6890](info): set up 2 sockets
Aug 11 12:14:17 dist-test-slave-nj21 krb5kdc[6890](info): commencing operation
krb5kdc: starting...
W20260811 12:14:19.102900 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:19 dist-test-slave-nj21 krb5kdc[6890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:19.387003 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34519
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:19.714046 6916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:19.714308 6916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:19.714437 6916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:19.721442 6916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:19.721549 6916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:19.721624 6916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:19.721704 6916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:19.721761 6916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:19.731196 6916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34519
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:19.732443 6916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:19.733702 6916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:19.744853 6923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:19.744921 6925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:19.744820 6933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:19.746074 6916 server_base.cc:1034] running on GCE node
I20260811 12:14:19.746791 6916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:19.748114 6916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:19.749306 6916 hybrid_clock.cc:648] HybridClock initialized: now 1786450459749266 us; error 31 us; skew 500 ppm
I20260811 12:14:19.749667 6916 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:19 dist-test-slave-nj21 krb5kdc[6890](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450459, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:19.752630 6916 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:19.754024 6916 webserver.cc:460] Webserver started at http://127.0.0.1:45579/ using document root <none> and password file <none>
I20260811 12:14:19.754667 6916 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:19.754783 6916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:19.755082 6916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:19.758119 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "00262deaec754da4893378993d1421fa"
format_stamp: "Formatted at 2026-08-11 12:14:19 on dist-test-slave-nj21"
server_key: "74313ee0efbd993c0e46021922b14d3a"
server_key_iv: "9a15ff77b147f390eb42739800a3e716"
server_key_version: "encryptionkey@0"
I20260811 12:14:19.758863 6916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "00262deaec754da4893378993d1421fa"
format_stamp: "Formatted at 2026-08-11 12:14:19 on dist-test-slave-nj21"
server_key: "74313ee0efbd993c0e46021922b14d3a"
server_key_iv: "9a15ff77b147f390eb42739800a3e716"
server_key_version: "encryptionkey@0"
I20260811 12:14:19.763738 6916 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 12:14:19.766918 6949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:19.768143 6916 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:14:19.768307 6916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "00262deaec754da4893378993d1421fa"
format_stamp: "Formatted at 2026-08-11 12:14:19 on dist-test-slave-nj21"
server_key: "74313ee0efbd993c0e46021922b14d3a"
server_key_iv: "9a15ff77b147f390eb42739800a3e716"
server_key_version: "encryptionkey@0"
I20260811 12:14:19.768507 6916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:19.784014 6916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:19.788056 6916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:19.788314 6916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:19.805752 6916 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45353
I20260811 12:14:19.805821 7031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45353 every 8 connection(s)
I20260811 12:14:19.807039 6916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:19.810655 7036 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:19.811944 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 6916
I20260811 12:14:19.812127 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:19.812682 4291 external_mini_cluster.cc:1428] Setting key 5e1b14cac597b316246c2833089b6710
I20260811 12:14:19.823122 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa: Bootstrap starting.
Aug 11 12:14:19 dist-test-slave-nj21 krb5kdc[6890](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:14:19.826066 7036 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:19.827070 7036 log.cc:824] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:19.830090 7036 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa: No bootstrap required, opened a new log
I20260811 12:14:19.834133 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:19.814393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33681 (local address 127.0.0.1:45353)
0811 12:14:19.815167 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:19.815181 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:19.815221 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:14:19.816864 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:19.816894 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:19.817344 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:19.817751 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:19.819210 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:19.820586 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:19.821859 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:19.822524 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:19.822687 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:19.826755 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:19.826763 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:19.826777 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:19.826822 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:19.830089 (+ 3267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:19.830759 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:19.830765 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:19.830771 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:19.830940 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:19.831308 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:19.831311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:19.831313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:19.831941 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:19.832238 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:19.832703 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 12:14:19.833138 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":539,"thread_start_us":323,"threads_started":1}
I20260811 12:14:19.837653 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:19.837884 7036 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260811 12:14:19.838228 7036 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:19.838315 7036 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00262deaec754da4893378993d1421fa, State: Initialized, Role: FOLLOWER
I20260811 12:14:19.838840 7036 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260811 12:14:19.839071 7036 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:19.839190 7036 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:19.839342 7036 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:19.841774 7036 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260811 12:14:19.842379 7036 leader_election.cc:302] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00262deaec754da4893378993d1421fa; no voters:
I20260811 12:14:19.843305 7036 leader_election.cc:288] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:19.843433 7050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:19.845058 7050 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [term 1 LEADER]: Becoming Leader. State: Replica: 00262deaec754da4893378993d1421fa, State: Running, Role: LEADER
I20260811 12:14:19.845270 7036 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:19.845642 7050 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } }
I20260811 12:14:19.848623 7054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00262deaec754da4893378993d1421fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260811 12:14:19.848997 7054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:19.852839 7055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00262deaec754da4893378993d1421fa. Latest consensus state: current_term: 1 leader_uuid: "00262deaec754da4893378993d1421fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00262deaec754da4893378993d1421fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45353 } } }
I20260811 12:14:19.853317 7055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:19.853840 7059 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:19.859287 7040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:19.842041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50665 (local address 127.0.0.1:45353)
0811 12:14:19.842185 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:19.842188 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:19.842641 (+ 453us) server_negotiation.cc:407] Connection header received
0811 12:14:19.842799 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:19.842805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:19.842871 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:19.842959 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:19.844824 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:19.845941 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:19.847607 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:19.848354 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:19.848487 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:19.851781 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:19.851788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:19.851793 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:19.851835 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:19.855047 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:19.855869 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:19.855875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:19.855891 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:19.856082 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:19.856555 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:19.856560 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:19.856563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:19.856974 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:19.858027 (+ 1053us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:19.858530 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 12:14:19.858842 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:14:19.860280 7059 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:19.874138 7059 catalog_manager.cc:1347] Generated new cluster ID: b19486a4bcd346f7bcb8fae8b7697639
I20260811 12:14:19.874261 7059 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:19.893743 7059 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:19.895011 7059 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:19.904215 7059 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00262deaec754da4893378993d1421fa: Generated new TSK 0
I20260811 12:14:19.905185 7059 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:19.917896 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 6916
I20260811 12:14:19.919087 7025 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:20.011662 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 6916
2026-08-11T12:14:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2138 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:20 dist-test-slave-nj21 krb5kdc[7082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:20 dist-test-slave-nj21 krb5kdc[7082](info): set up 2 sockets
Aug 11 12:14:20 dist-test-slave-nj21 krb5kdc[7082](info): commencing operation
krb5kdc: starting...
W20260811 12:14:21.241255 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:21 dist-test-slave-nj21 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:21.516515 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:21.847395 7100 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:21.847647 7100 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:21.847790 7100 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:21.854765 7100 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:21.854882 7100 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:21.854952 7100 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:21.855039 7100 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:21.855094 7100 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:21.864480 7100 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:21.865602 7100 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:21.866811 7100 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:21.878235 7107 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:21.878301 7108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:21.878266 7116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:21.879204 7100 server_base.cc:1034] running on GCE node
I20260811 12:14:21.879995 7100 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:21.881246 7100 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:21.882436 7100 hybrid_clock.cc:648] HybridClock initialized: now 1786450461882398 us; error 33 us; skew 500 ppm
I20260811 12:14:21.882757 7100 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:21 dist-test-slave-nj21 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450461, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:21.885632 7100 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:21.886943 7100 webserver.cc:460] Webserver started at http://127.0.0.1:46353/ using document root <none> and password file <none>
I20260811 12:14:21.887578 7100 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:21.887706 7100 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:21.887995 7100 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:21.890820 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "706e8cc548454887aa5b7ce323acd6f8"
format_stamp: "Formatted at 2026-08-11 12:14:21 on dist-test-slave-nj21"
server_key: "3585685d73390712e650312ae6615261"
server_key_iv: "a879697cbdb91df39e88e39ec7cb55ea"
server_key_version: "encryptionkey@0"
I20260811 12:14:21.891542 7100 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "706e8cc548454887aa5b7ce323acd6f8"
format_stamp: "Formatted at 2026-08-11 12:14:21 on dist-test-slave-nj21"
server_key: "3585685d73390712e650312ae6615261"
server_key_iv: "a879697cbdb91df39e88e39ec7cb55ea"
server_key_version: "encryptionkey@0"
I20260811 12:14:21.896276 7100 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 12:14:21.899411 7128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:21.900625 7100 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:14:21.900765 7100 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "706e8cc548454887aa5b7ce323acd6f8"
format_stamp: "Formatted at 2026-08-11 12:14:21 on dist-test-slave-nj21"
server_key: "3585685d73390712e650312ae6615261"
server_key_iv: "a879697cbdb91df39e88e39ec7cb55ea"
server_key_version: "encryptionkey@0"
I20260811 12:14:21.900929 7100 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:21.936035 7100 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:21.939667 7100 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:21.939939 7100 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:21.956740 7100 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35049
I20260811 12:14:21.956771 7212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35049 every 8 connection(s)
I20260811 12:14:21.958001 7100 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:21.961670 7213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:21.962014 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7100
I20260811 12:14:21.962193 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:21.962774 4291 external_mini_cluster.cc:1428] Setting key 1faf427759132d38cc7a1b00cc4b784b
I20260811 12:14:21.975126 7213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8: Bootstrap starting.
I20260811 12:14:21.975248 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:21.964471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55173 (local address 127.0.0.1:35049)
0811 12:14:21.965253 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:21.965264 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:21.965305 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:21.966794 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:21.966814 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:21.967185 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:21.967486 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:21.968117 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:21.969407 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:21.971188 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:21.971883 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:21.972008 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:21.973198 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:21.973205 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:21.973220 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:21.973714 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:21.973789 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:21.973813 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:21.973904 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:21.974116 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:14:21.974325 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":361,"threads_started":1}
I20260811 12:14:21.978648 7213 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:21.979039 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:21.979892 7213 log.cc:824] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:21.982968 7213 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8: No bootstrap required, opened a new log
I20260811 12:14:21.989957 7216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:21.983873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54225 (local address 127.0.0.1:35049)
0811 12:14:21.984035 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:21.984040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:21.984444 (+ 404us) server_negotiation.cc:407] Connection header received
0811 12:14:21.984558 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:21.984563 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:21.984645 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:21.984746 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:21.985194 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:21.986267 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:21.987521 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:21.988247 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:21.988395 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:21.989246 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:21.989253 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:21.989257 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:21.989278 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:21.989329 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:21.989348 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:21.989473 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:21.989540 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 12:14:21.989620 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 12:14:21.992033 7213 raft_consensus.cc:348] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260811 12:14:21.992347 7213 raft_consensus.cc:374] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:21.992429 7213 raft_consensus.cc:729] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 706e8cc548454887aa5b7ce323acd6f8, State: Initialized, Role: FOLLOWER
I20260811 12:14:21.992987 7213 consensus_queue.cc:260] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260811 12:14:21.993219 7213 raft_consensus.cc:388] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:21.993341 7213 raft_consensus.cc:482] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:21.993494 7213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:21.995743 7213 raft_consensus.cc:504] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260811 12:14:21.996199 7213 leader_election.cc:302] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 706e8cc548454887aa5b7ce323acd6f8; no voters:
I20260811 12:14:21.996743 7213 leader_election.cc:288] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:21.996865 7228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:21.997488 7228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [term 1 LEADER]: Becoming Leader. State: Replica: 706e8cc548454887aa5b7ce323acd6f8, State: Running, Role: LEADER
I20260811 12:14:21.997954 7228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } }
I20260811 12:14:21.998669 7213 sys_catalog.cc:564] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:22.000818 7230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "706e8cc548454887aa5b7ce323acd6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260811 12:14:22.000859 7232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 706e8cc548454887aa5b7ce323acd6f8. Latest consensus state: current_term: 1 leader_uuid: "706e8cc548454887aa5b7ce323acd6f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "706e8cc548454887aa5b7ce323acd6f8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35049 } } }
I20260811 12:14:22.001264 7232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:22.001261 7230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:22.002760 7241 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:22.007786 7241 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:22.016459 7241 catalog_manager.cc:1347] Generated new cluster ID: 4195b3eb7eea42b3b64152114901b1b9
I20260811 12:14:22.016556 7241 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:22.029053 7241 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:22.030196 7241 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:22.040900 7241 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 706e8cc548454887aa5b7ce323acd6f8: Generated new TSK 0
I20260811 12:14:22.041608 7241 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:22.051307 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7100
W20260811 12:14:22.052417 7205 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55173
W20260811 12:14:22.053098 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7100: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:22.053215 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7100
2026-08-11T12:14:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2040 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:22 dist-test-slave-nj21 krb5kdc[7263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:22 dist-test-slave-nj21 krb5kdc[7263](info): set up 2 sockets
Aug 11 12:14:22 dist-test-slave-nj21 krb5kdc[7263](info): commencing operation
krb5kdc: starting...
W20260811 12:14:23.284834 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.190s user 0.006s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:23 dist-test-slave-nj21 krb5kdc[7263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:23.559226 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42693
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:23.889359 7279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:23.889635 7279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:23.889765 7279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:23.896862 7279 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:23.896963 7279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:23.897039 7279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:23.897122 7279 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:23.897189 7279 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:23.906908 7279 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:23.908097 7279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:23.909314 7279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:23.920405 7289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:23.921075 7287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:23.921108 7291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:23.921531 7279 server_base.cc:1034] running on GCE node
I20260811 12:14:23.922202 7279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:23.923489 7279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:23.924683 7279 hybrid_clock.cc:648] HybridClock initialized: now 1786450463924634 us; error 37 us; skew 500 ppm
I20260811 12:14:23.925005 7279 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:23 dist-test-slave-nj21 krb5kdc[7263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450463, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:23.931892 7279 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:23.933245 7279 webserver.cc:460] Webserver started at http://127.0.0.1:33017/ using document root <none> and password file <none>
I20260811 12:14:23.933894 7279 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:23.934010 7279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:23.934301 7279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:23.937222 7279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "98078001aaff4d118342da93ace2ad80"
format_stamp: "Formatted at 2026-08-11 12:14:23 on dist-test-slave-nj21"
server_key: "80f3f3a45420fe1c061188ea5dfc66fc"
server_key_iv: "418975515b10fa7ac2a4b27140f6afe2"
server_key_version: "encryptionkey@0"
I20260811 12:14:23.937954 7279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "98078001aaff4d118342da93ace2ad80"
format_stamp: "Formatted at 2026-08-11 12:14:23 on dist-test-slave-nj21"
server_key: "80f3f3a45420fe1c061188ea5dfc66fc"
server_key_iv: "418975515b10fa7ac2a4b27140f6afe2"
server_key_version: "encryptionkey@0"
I20260811 12:14:23.942785 7279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 12:14:23.946017 7301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:23.947213 7279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:14:23.947374 7279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "98078001aaff4d118342da93ace2ad80"
format_stamp: "Formatted at 2026-08-11 12:14:23 on dist-test-slave-nj21"
server_key: "80f3f3a45420fe1c061188ea5dfc66fc"
server_key_iv: "418975515b10fa7ac2a4b27140f6afe2"
server_key_version: "encryptionkey@0"
I20260811 12:14:23.947544 7279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:23.952951 7279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:23.956482 7279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:23.956740 7279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:23.973520 7279 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42693
I20260811 12:14:23.973573 7397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42693 every 8 connection(s)
I20260811 12:14:23.974771 7279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:23.978245 7398 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:23.984202 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7279
I20260811 12:14:23.984339 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:23.984792 4291 external_mini_cluster.cc:1428] Setting key aad9d98e7e0ad4362c3ba2c077d64cd6
I20260811 12:14:23.988684 7398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80: Bootstrap starting.
I20260811 12:14:23.991874 7398 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:23.992089 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:23.986470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39517 (local address 127.0.0.1:42693)
0811 12:14:23.987220 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:23.987231 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:23.987271 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:14:23.988927 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:23.988949 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:23.989247 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:23.989582 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:23.990570 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:23.990577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:23.990590 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:23.990604 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:23.990660 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:23.990683 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:23.990888 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:23.991020 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 12:14:23.991187 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":329,"threads_started":1}
I20260811 12:14:23.993140 7398 log.cc:824] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:23.995908 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:23.996044 7398 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80: No bootstrap required, opened a new log
I20260811 12:14:24.003197 7405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:24.000875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36249 (local address 127.0.0.1:42693)
0811 12:14:24.001026 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:24.001031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:24.001491 (+ 460us) server_negotiation.cc:407] Connection header received
0811 12:14:24.001634 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:24.001637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:24.001685 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:24.001761 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:24.002603 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:24.002607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:24.002610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:24.002624 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:24.002662 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:24.002676 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:24.002772 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:24.002888 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 12:14:24.002932 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 12:14:24.005257 7398 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260811 12:14:24.005576 7398 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:24.005669 7398 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98078001aaff4d118342da93ace2ad80, State: Initialized, Role: FOLLOWER
I20260811 12:14:24.006177 7398 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260811 12:14:24.006403 7398 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:24.006526 7398 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:24.006680 7398 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:24.008980 7398 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260811 12:14:24.009455 7398 leader_election.cc:302] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98078001aaff4d118342da93ace2ad80; no voters:
I20260811 12:14:24.009953 7398 leader_election.cc:288] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:24.010075 7415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:24.010730 7415 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [term 1 LEADER]: Becoming Leader. State: Replica: 98078001aaff4d118342da93ace2ad80, State: Running, Role: LEADER
I20260811 12:14:24.011281 7415 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } }
I20260811 12:14:24.011818 7398 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:24.013865 7416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98078001aaff4d118342da93ace2ad80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } } }
I20260811 12:14:24.013945 7417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98078001aaff4d118342da93ace2ad80. Latest consensus state: current_term: 1 leader_uuid: "98078001aaff4d118342da93ace2ad80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98078001aaff4d118342da93ace2ad80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42693 } } }
I20260811 12:14:24.014309 7416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:24.014309 7417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:24.017158 7428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:24.022333 7428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:24.029820 7428 catalog_manager.cc:1347] Generated new cluster ID: 4a2a4b46c60b46938b29ee5ef7afaf6f
I20260811 12:14:24.029927 7428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:24.040556 7428 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:24.041014 7428 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:24.053982 7428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98078001aaff4d118342da93ace2ad80: Generated new TSK 0
I20260811 12:14:24.054850 7428 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:24.071799 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7279
W20260811 12:14:24.072839 7391 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39517
W20260811 12:14:24.073490 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7279: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:24.073606 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7279
2026-08-11T12:14:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2026 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:24 dist-test-slave-nj21 krb5kdc[7452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:24 dist-test-slave-nj21 krb5kdc[7452](info): set up 2 sockets
Aug 11 12:14:24 dist-test-slave-nj21 krb5kdc[7452](info): commencing operation
krb5kdc: starting...
W20260811 12:14:25.325439 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:25 dist-test-slave-nj21 krb5kdc[7452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:25.608906 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39077
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:25.939204 7477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:25.939484 7477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:25.939620 7477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:25.946580 7477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:25.946689 7477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:25.946774 7477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:25.946853 7477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:25.946909 7477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:25.956354 7477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:25.957561 7477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:25.958827 7477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:25.969803 7490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:25.969846 7487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:25.969885 7486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:25.970604 7477 server_base.cc:1034] running on GCE node
I20260811 12:14:25.971280 7477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:25.972563 7477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:25.973766 7477 hybrid_clock.cc:648] HybridClock initialized: now 1786450465973724 us; error 24 us; skew 500 ppm
I20260811 12:14:25.974080 7477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:25 dist-test-slave-nj21 krb5kdc[7452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:25.977082 7477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:25.978415 7477 webserver.cc:460] Webserver started at http://127.0.0.1:45615/ using document root <none> and password file <none>
I20260811 12:14:25.979053 7477 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:25.979168 7477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:25.979455 7477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:25.982302 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "360cf9cb885c42c6bea554a4b2b3d516"
format_stamp: "Formatted at 2026-08-11 12:14:25 on dist-test-slave-nj21"
server_key: "7120cb5bcb8349794dce2dd6aefc8d8d"
server_key_iv: "3cd42b5c13a30330ee98db0b420e373f"
server_key_version: "encryptionkey@0"
I20260811 12:14:25.983023 7477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "360cf9cb885c42c6bea554a4b2b3d516"
format_stamp: "Formatted at 2026-08-11 12:14:25 on dist-test-slave-nj21"
server_key: "7120cb5bcb8349794dce2dd6aefc8d8d"
server_key_iv: "3cd42b5c13a30330ee98db0b420e373f"
server_key_version: "encryptionkey@0"
I20260811 12:14:25.987960 7477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 12:14:25.991264 7501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:25.992475 7477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:25.992630 7477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "360cf9cb885c42c6bea554a4b2b3d516"
format_stamp: "Formatted at 2026-08-11 12:14:25 on dist-test-slave-nj21"
server_key: "7120cb5bcb8349794dce2dd6aefc8d8d"
server_key_iv: "3cd42b5c13a30330ee98db0b420e373f"
server_key_version: "encryptionkey@0"
I20260811 12:14:25.992805 7477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:26.006356 7477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:26.010025 7477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:26.010282 7477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:26.027892 7477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39077
I20260811 12:14:26.027916 7592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39077 every 8 connection(s)
I20260811 12:14:26.029356 7477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:26.033228 7593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:26.033418 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7477
I20260811 12:14:26.033594 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:26.034094 4291 external_mini_cluster.cc:1428] Setting key 5b0ae171e1a9635367e407fc84d6a7a7
I20260811 12:14:26.046350 7593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516: Bootstrap starting.
Aug 11 12:14:26 dist-test-slave-nj21 krb5kdc[7452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:14:26.049940 7593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:26.050949 7593 log.cc:824] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:26.053733 7593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516: No bootstrap required, opened a new log
I20260811 12:14:26.058079 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:26.036049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:59353 (local address 127.0.0.1:39077)
0811 12:14:26.036853 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:26.036864 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:26.036911 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:14:26.039006 (+ 2095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:26.039033 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:26.039459 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:26.039825 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:26.041583 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:26.043016 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:26.044791 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:26.045478 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:26.045616 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:26.050485 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:26.050493 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:26.050508 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:26.050546 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:26.053770 (+ 3224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:26.054733 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:26.054761 (+ 28us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:26.054767 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:26.054938 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:26.055410 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:26.055416 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:26.055418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:26.056038 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:26.056279 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:26.056774 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 12:14:26.057074 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":581,"thread_start_us":335,"threads_started":1}
I20260811 12:14:26.062001 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:26.062549 7593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260811 12:14:26.062939 7593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:26.063031 7593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360cf9cb885c42c6bea554a4b2b3d516, State: Initialized, Role: FOLLOWER
I20260811 12:14:26.063621 7593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260811 12:14:26.063946 7593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:26.064069 7593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:26.064213 7593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:26.067073 7593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260811 12:14:26.067549 7593 leader_election.cc:302] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 360cf9cb885c42c6bea554a4b2b3d516; no voters:
I20260811 12:14:26.068187 7593 leader_election.cc:288] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:26.068331 7608 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:26.069005 7608 raft_consensus.cc:686] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [term 1 LEADER]: Becoming Leader. State: Replica: 360cf9cb885c42c6bea554a4b2b3d516, State: Running, Role: LEADER
I20260811 12:14:26.069671 7608 consensus_queue.cc:237] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } }
I20260811 12:14:26.071563 7593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:26.072377 7614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "360cf9cb885c42c6bea554a4b2b3d516" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } } }
I20260811 12:14:26.072757 7614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:26.073243 7611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 360cf9cb885c42c6bea554a4b2b3d516. Latest consensus state: current_term: 1 leader_uuid: "360cf9cb885c42c6bea554a4b2b3d516" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "360cf9cb885c42c6bea554a4b2b3d516" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39077 } } }
I20260811 12:14:26.073557 7611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:26.076241 7623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:26.082041 7623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:26.085024 7596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:26.066996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:56131 (local address 127.0.0.1:39077)
0811 12:14:26.067171 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:26.067177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:26.067587 (+ 410us) server_negotiation.cc:407] Connection header received
0811 12:14:26.067794 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:26.067799 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:26.067874 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:26.067976 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:26.069845 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:26.071027 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:26.072806 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:26.073581 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:26.073704 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:26.077014 (+ 3310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:26.077021 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:26.077025 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:26.077062 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:26.080656 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:26.081647 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:26.081651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:26.081653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:26.081772 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:26.082467 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:26.082472 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:26.082475 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:26.082887 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:26.084266 (+ 1379us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:26.084462 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 12:14:26.084637 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:14:26.094379 7623 catalog_manager.cc:1347] Generated new cluster ID: 2a9a779856ab4a5cb336cb35e6ceebc1
I20260811 12:14:26.094492 7623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:26.138147 7623 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:26.139568 7623 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:26.152853 7623 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 360cf9cb885c42c6bea554a4b2b3d516: Generated new TSK 0
I20260811 12:14:26.153692 7623 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:26.167816 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7477
I20260811 12:14:26.168906 7586 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:26.261227 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7477
W20260811 12:14:26.286175 7447 connection.cc:535] client connection to 127.0.0.1:39077 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39077): Connection reset by peer (error 104)
2026-08-11T12:14:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2184 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:26 dist-test-slave-nj21 krb5kdc[7648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:26 dist-test-slave-nj21 krb5kdc[7648](info): set up 2 sockets
Aug 11 12:14:26 dist-test-slave-nj21 krb5kdc[7648](info): commencing operation
krb5kdc: starting...
W20260811 12:14:27.517321 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:27 dist-test-slave-nj21 krb5kdc[7648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:27.794050 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:28.120754 7673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:28.121025 7673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:28.121156 7673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:28.128026 7673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:28.128124 7673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:28.128202 7673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:28.128295 7673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:28.128360 7673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:28.137835 7673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:28.139074 7673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:28.140362 7673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:28.151578 7680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:28.151615 7684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:28.151535 7682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:28.152415 7673 server_base.cc:1034] running on GCE node
I20260811 12:14:28.153198 7673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:28.154532 7673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:28.155748 7673 hybrid_clock.cc:648] HybridClock initialized: now 1786450468155701 us; error 37 us; skew 500 ppm
I20260811 12:14:28.156091 7673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:28 dist-test-slave-nj21 krb5kdc[7648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:28.159039 7673 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:28.160435 7673 webserver.cc:460] Webserver started at http://127.0.0.1:44931/ using document root <none> and password file <none>
I20260811 12:14:28.161082 7673 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:28.161181 7673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:28.161434 7673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:28.164312 7673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "4a740d26aadb4d99aa48628b668821ac"
format_stamp: "Formatted at 2026-08-11 12:14:28 on dist-test-slave-nj21"
server_key: "779faa44fc87709904ab53ec83e05c7e"
server_key_iv: "fa92b442eb45838c9dd155942c04c9be"
server_key_version: "encryptionkey@0"
I20260811 12:14:28.165020 7673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "4a740d26aadb4d99aa48628b668821ac"
format_stamp: "Formatted at 2026-08-11 12:14:28 on dist-test-slave-nj21"
server_key: "779faa44fc87709904ab53ec83e05c7e"
server_key_iv: "fa92b442eb45838c9dd155942c04c9be"
server_key_version: "encryptionkey@0"
I20260811 12:14:28.169936 7673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:14:28.173278 7696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:28.174530 7673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 12:14:28.174674 7673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "4a740d26aadb4d99aa48628b668821ac"
format_stamp: "Formatted at 2026-08-11 12:14:28 on dist-test-slave-nj21"
server_key: "779faa44fc87709904ab53ec83e05c7e"
server_key_iv: "fa92b442eb45838c9dd155942c04c9be"
server_key_version: "encryptionkey@0"
I20260811 12:14:28.174829 7673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:28.193428 7673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:28.197284 7673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:28.197515 7673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:28.215063 7673 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44863
I20260811 12:14:28.215101 7796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44863 every 8 connection(s)
I20260811 12:14:28.216403 7673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:28.216771 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7673
I20260811 12:14:28.216944 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:28.217485 4291 external_mini_cluster.cc:1428] Setting key 5db5806ed6ad5ab32e8179c6a9ca7654
I20260811 12:14:28.220679 7797 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:28.230039 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:28.219306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:41053 (local address 127.0.0.1:44863)
0811 12:14:28.220080 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:28.220092 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:28.220131 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:14:28.221768 (+ 1637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:28.221801 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:28.222162 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:28.222486 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:28.223140 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:28.224512 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:28.225881 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:28.226505 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:28.226675 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:28.227911 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:28.227919 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:28.227931 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:28.228448 (+ 517us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:28.228522 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:28.228547 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:28.228691 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:28.228991 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 12:14:28.229137 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":355,"threads_started":1}
I20260811 12:14:28.231477 7797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac: Bootstrap starting.
I20260811 12:14:28.234050 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:28.234656 7797 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:28.235612 7797 log.cc:824] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:28.238583 7797 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac: No bootstrap required, opened a new log
I20260811 12:14:28.245157 7803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:28.238677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:33771 (local address 127.0.0.1:44863)
0811 12:14:28.238825 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:28.238831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:28.239294 (+ 463us) server_negotiation.cc:407] Connection header received
0811 12:14:28.239497 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:28.239502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:28.239564 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:28.239650 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:28.240254 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:28.241125 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:28.242503 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:28.243033 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:28.243158 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:28.244417 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:28.244422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:28.244425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:28.244441 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:28.244499 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:28.244518 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:28.244614 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:28.244831 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 12:14:28.244893 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:14:28.246451 7797 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260811 12:14:28.246829 7797 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:28.246924 7797 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a740d26aadb4d99aa48628b668821ac, State: Initialized, Role: FOLLOWER
I20260811 12:14:28.247498 7797 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260811 12:14:28.247740 7797 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:28.247877 7797 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:28.248042 7797 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:28.250128 7797 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260811 12:14:28.250603 7797 leader_election.cc:302] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a740d26aadb4d99aa48628b668821ac; no voters:
I20260811 12:14:28.251170 7797 leader_election.cc:288] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:28.251295 7812 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:28.251950 7812 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [term 1 LEADER]: Becoming Leader. State: Replica: 4a740d26aadb4d99aa48628b668821ac, State: Running, Role: LEADER
I20260811 12:14:28.252565 7812 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } }
I20260811 12:14:28.253352 7797 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:28.255262 7813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a740d26aadb4d99aa48628b668821ac. Latest consensus state: current_term: 1 leader_uuid: "4a740d26aadb4d99aa48628b668821ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } } }
I20260811 12:14:28.255694 7813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:28.256438 7814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a740d26aadb4d99aa48628b668821ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a740d26aadb4d99aa48628b668821ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44863 } } }
I20260811 12:14:28.256716 7814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:28.260850 7824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:28.265964 7824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:28.274761 7824 catalog_manager.cc:1347] Generated new cluster ID: 56fd01844ab5473786714614a7b094d5
I20260811 12:14:28.274854 7824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:28.288827 7824 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:28.290202 7824 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:28.299140 7824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a740d26aadb4d99aa48628b668821ac: Generated new TSK 0
I20260811 12:14:28.299962 7824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:28.313958 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7673
W20260811 12:14:28.315004 7789 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:41053
W20260811 12:14:28.315688 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7673: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:28.315821 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7673
W20260811 12:14:28.342065 7646 connection.cc:535] client connection to 127.0.0.1:44863 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44863): Connection reset by peer (error 104)
2026-08-11T12:14:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:28 dist-test-slave-nj21 krb5kdc[7840](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:28 dist-test-slave-nj21 krb5kdc[7840](info): set up 2 sockets
Aug 11 12:14:28 dist-test-slave-nj21 krb5kdc[7840](info): commencing operation
krb5kdc: starting...
W20260811 12:14:29.560309 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.200s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:29 dist-test-slave-nj21 krb5kdc[7840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:29.836659 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33615
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33615
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:30.169550 7860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:30.169827 7860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:30.169960 7860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:30.177069 7860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:30.177173 7860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:30.177248 7860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:30.177326 7860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:30.177383 7860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:30.186800 7860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33615
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:30.188062 7860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:30.189317 7860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:30.200294 7872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:30.200227 7873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:30.200366 7876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:30.201088 7860 server_base.cc:1034] running on GCE node
I20260811 12:14:30.201769 7860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:30.203048 7860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:30.204233 7860 hybrid_clock.cc:648] HybridClock initialized: now 1786450470204186 us; error 35 us; skew 500 ppm
I20260811 12:14:30.204537 7860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:30 dist-test-slave-nj21 krb5kdc[7840](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:30.212446 7860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:30.213865 7860 webserver.cc:460] Webserver started at http://127.0.0.1:46715/ using document root <none> and password file <none>
I20260811 12:14:30.214500 7860 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:30.214614 7860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:30.214915 7860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:30.217816 7860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "fa3a15fd5c6748fba042997a27e5f35c"
format_stamp: "Formatted at 2026-08-11 12:14:30 on dist-test-slave-nj21"
server_key: "593456796ac0a648fdde90785b7ac790"
server_key_iv: "483b8bb9b84de211df711f3282fa8b63"
server_key_version: "encryptionkey@0"
I20260811 12:14:30.218533 7860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "fa3a15fd5c6748fba042997a27e5f35c"
format_stamp: "Formatted at 2026-08-11 12:14:30 on dist-test-slave-nj21"
server_key: "593456796ac0a648fdde90785b7ac790"
server_key_iv: "483b8bb9b84de211df711f3282fa8b63"
server_key_version: "encryptionkey@0"
I20260811 12:14:30.223487 7860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 12:14:30.226780 7888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:30.228060 7860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:30.228221 7860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "fa3a15fd5c6748fba042997a27e5f35c"
format_stamp: "Formatted at 2026-08-11 12:14:30 on dist-test-slave-nj21"
server_key: "593456796ac0a648fdde90785b7ac790"
server_key_iv: "483b8bb9b84de211df711f3282fa8b63"
server_key_version: "encryptionkey@0"
I20260811 12:14:30.228397 7860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:30.233855 7860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:30.237547 7860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:30.237823 7860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:30.255344 7860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33615
I20260811 12:14:30.255400 7974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33615 every 8 connection(s)
I20260811 12:14:30.256687 7860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:30.260512 7975 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:30.260545 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 7860
I20260811 12:14:30.260680 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:30.261173 4291 external_mini_cluster.cc:1428] Setting key 731e7c5340ea8c62d7f4ba527150edba
I20260811 12:14:30.269515 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:30.262881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:56261 (local address 127.0.0.1:33615)
0811 12:14:30.263851 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:30.263871 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:30.263925 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:14:30.265642 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:30.265672 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:30.266105 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:30.266610 (+ 505us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:30.267588 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:30.267596 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:30.267611 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:30.267632 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:30.267770 (+ 138us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:30.267806 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:30.268027 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:30.268165 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 12:14:30.268357 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":362,"threads_started":1}
I20260811 12:14:30.272898 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:30.275130 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c: Bootstrap starting.
I20260811 12:14:30.278800 7975 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:30.279956 7975 log.cc:824] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:30.280129 7981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:30.277994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:35379 (local address 127.0.0.1:33615)
0811 12:14:30.278146 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:30.278152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:30.278429 (+ 277us) server_negotiation.cc:407] Connection header received
0811 12:14:30.278545 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:30.278550 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:30.278611 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:30.278721 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:30.279488 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:30.279494 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:30.279498 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:30.279515 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:30.279558 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:30.279583 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:30.279719 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:30.279780 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 12:14:30.279832 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:14:30.282671 7975 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c: No bootstrap required, opened a new log
I20260811 12:14:30.290189 7975 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } }
I20260811 12:14:30.290589 7975 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:30.290694 7975 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3a15fd5c6748fba042997a27e5f35c, State: Initialized, Role: FOLLOWER
I20260811 12:14:30.291239 7975 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } }
I20260811 12:14:30.291446 7975 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:30.291571 7975 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:30.291747 7975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:30.293754 7975 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } }
I20260811 12:14:30.294201 7975 leader_election.cc:302] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa3a15fd5c6748fba042997a27e5f35c; no voters:
I20260811 12:14:30.294736 7975 leader_election.cc:288] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:30.294863 7989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:30.295496 7989 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [term 1 LEADER]: Becoming Leader. State: Replica: fa3a15fd5c6748fba042997a27e5f35c, State: Running, Role: LEADER
I20260811 12:14:30.296046 7989 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } }
I20260811 12:14:30.296622 7975 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:30.299513 7991 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa3a15fd5c6748fba042997a27e5f35c. Latest consensus state: current_term: 1 leader_uuid: "fa3a15fd5c6748fba042997a27e5f35c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } } }
I20260811 12:14:30.299979 7991 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:30.302870 7990 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa3a15fd5c6748fba042997a27e5f35c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa3a15fd5c6748fba042997a27e5f35c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33615 } } }
I20260811 12:14:30.303155 7990 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:30.303375 8008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:30.308554 8008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:30.316354 8008 catalog_manager.cc:1347] Generated new cluster ID: 1eefbd07e4c84ef5a748d5850f3d0da2
I20260811 12:14:30.316457 8008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:30.366225 8008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:30.366654 8008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:30.372424 8008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa3a15fd5c6748fba042997a27e5f35c: Generated new TSK 0
I20260811 12:14:30.373142 8008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:30.392688 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7860
W20260811 12:14:30.393842 7970 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:56261
W20260811 12:14:30.394520 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 7860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:30.394654 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 7860
2026-08-11T12:14:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2078 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:30 dist-test-slave-nj21 krb5kdc[8036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:30 dist-test-slave-nj21 krb5kdc[8036](info): set up 2 sockets
Aug 11 12:14:30 dist-test-slave-nj21 krb5kdc[8036](info): commencing operation
krb5kdc: starting...
W20260811 12:14:31.633767 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.195s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:31 dist-test-slave-nj21 krb5kdc[8036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:31.924883 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:32.257870 8061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:32.258134 8061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:32.258273 8061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:32.265324 8061 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:32.265425 8061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:32.265502 8061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:32.265580 8061 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:32.265638 8061 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:32.275063 8061 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:32.276314 8061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:32.277590 8061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:32.288676 8072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:32.288789 8073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:32.289026 8077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:32.289748 8061 server_base.cc:1034] running on GCE node
I20260811 12:14:32.290503 8061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:32.291854 8061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:32.293071 8061 hybrid_clock.cc:648] HybridClock initialized: now 1786450472293024 us; error 37 us; skew 500 ppm
I20260811 12:14:32.293416 8061 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:32 dist-test-slave-nj21 krb5kdc[8036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:32.296386 8061 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:32.298054 8061 webserver.cc:460] Webserver started at http://127.0.0.1:42437/ using document root <none> and password file <none>
I20260811 12:14:32.298704 8061 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:32.298828 8061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:32.299118 8061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:32.302052 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "d6487ea5eca6476a93d18717a693e04c"
format_stamp: "Formatted at 2026-08-11 12:14:32 on dist-test-slave-nj21"
server_key: "bf1fed085bb4e90899f1f41457c9b2e1"
server_key_iv: "4851650605d28b5b1d32bc1de2d5d101"
server_key_version: "encryptionkey@0"
I20260811 12:14:32.302814 8061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "d6487ea5eca6476a93d18717a693e04c"
format_stamp: "Formatted at 2026-08-11 12:14:32 on dist-test-slave-nj21"
server_key: "bf1fed085bb4e90899f1f41457c9b2e1"
server_key_iv: "4851650605d28b5b1d32bc1de2d5d101"
server_key_version: "encryptionkey@0"
I20260811 12:14:32.307638 8061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 12:14:32.310951 8088 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:32.312180 8061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:32.312335 8061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "d6487ea5eca6476a93d18717a693e04c"
format_stamp: "Formatted at 2026-08-11 12:14:32 on dist-test-slave-nj21"
server_key: "bf1fed085bb4e90899f1f41457c9b2e1"
server_key_iv: "4851650605d28b5b1d32bc1de2d5d101"
server_key_version: "encryptionkey@0"
I20260811 12:14:32.312507 8061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:32.339512 8061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:32.343269 8061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:32.343533 8061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:32.361027 8061 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45381
I20260811 12:14:32.361063 8179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45381 every 8 connection(s)
I20260811 12:14:32.362327 8061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:32.363049 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8061
I20260811 12:14:32.363219 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:32.363807 4291 external_mini_cluster.cc:1428] Setting key 9535c722719ec322b3dbde3e7de398cb
I20260811 12:14:32.366899 8181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 12:14:32 dist-test-slave-nj21 krb5kdc[8036](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 12:14:32.379535 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c: Bootstrap starting.
I20260811 12:14:32.382558 8181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:32.383800 8181 log.cc:824] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:32.386925 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:32.365575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:56943 (local address 127.0.0.1:45381)
0811 12:14:32.366350 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:32.366365 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:32.366419 (+ 54us) server_negotiation.cc:407] Connection header received
0811 12:14:32.368712 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:32.368751 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:32.369232 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:32.369672 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:32.370928 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:32.373046 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:32.374254 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:32.375234 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:32.375414 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:32.379624 (+ 4210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:32.379634 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:32.379653 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:32.379720 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:32.382538 (+ 2818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:32.383451 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:32.383458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:32.383463 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:32.383632 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:32.384192 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:32.384195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:32.384198 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:32.384974 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:32.385280 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:32.385684 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 12:14:32.385941 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":306,"threads_started":1}
I20260811 12:14:32.386947 8181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c: No bootstrap required, opened a new log
I20260811 12:14:32.390873 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:32.396641 8181 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260811 12:14:32.397080 8181 raft_consensus.cc:374] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:32.397179 8181 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6487ea5eca6476a93d18717a693e04c, State: Initialized, Role: FOLLOWER
I20260811 12:14:32.397756 8181 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260811 12:14:32.397964 8181 raft_consensus.cc:388] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:32.398092 8181 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:32.398255 8181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:32.400950 8181 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260811 12:14:32.401559 8181 leader_election.cc:302] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d6487ea5eca6476a93d18717a693e04c; no voters:
I20260811 12:14:32.402298 8181 leader_election.cc:288] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:32.402458 8200 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:32.403074 8200 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [term 1 LEADER]: Becoming Leader. State: Replica: d6487ea5eca6476a93d18717a693e04c, State: Running, Role: LEADER
I20260811 12:14:32.403793 8200 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260811 12:14:32.405596 8181 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:32.407476 8205 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d6487ea5eca6476a93d18717a693e04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } } }
I20260811 12:14:32.407940 8205 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:32.408829 8220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:32.410284 8204 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6487ea5eca6476a93d18717a693e04c. Latest consensus state: current_term: 1 leader_uuid: "d6487ea5eca6476a93d18717a693e04c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d6487ea5eca6476a93d18717a693e04c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } } }
I20260811 12:14:32.410626 8204 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:32.414322 8185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:32.395939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:37579 (local address 127.0.0.1:45381)
0811 12:14:32.396107 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:32.396113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:32.396547 (+ 434us) server_negotiation.cc:407] Connection header received
0811 12:14:32.396674 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:32.396680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:32.396763 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:32.396867 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:32.398286 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:32.399433 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:32.400688 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:32.401339 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:32.401460 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:32.404525 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:32.404531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:32.404535 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:32.404570 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:32.409978 (+ 5408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:32.411003 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:32.411012 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:32.411020 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:32.411183 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:32.411833 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:32.411838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:32.411842 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:32.412244 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:32.412556 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:32.413693 (+ 1137us) server_negotiation.cc:299] Negotiation successful
0811 12:14:32.413901 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":54}
I20260811 12:14:32.414676 8220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:32.424916 8220 catalog_manager.cc:1347] Generated new cluster ID: 060206e05a1e42f78eb4fc691a379112
I20260811 12:14:32.425077 8220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:32.445581 8220 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:32.446768 8220 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:32.460398 8220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6487ea5eca6476a93d18717a693e04c: Generated new TSK 0
I20260811 12:14:32.461154 8220 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:32.466599 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8061
I20260811 12:14:32.467702 8173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:32.556290 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8061
W20260811 12:14:32.580957 8032 connection.cc:535] client connection to 127.0.0.1:45381 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45381): Connection reset by peer (error 104)
2026-08-11T12:14:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2160 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:32 dist-test-slave-nj21 krb5kdc[8243](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:32 dist-test-slave-nj21 krb5kdc[8243](info): set up 2 sockets
Aug 11 12:14:32 dist-test-slave-nj21 krb5kdc[8243](info): commencing operation
krb5kdc: starting...
W20260811 12:14:33.798641 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.006s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:33 dist-test-slave-nj21 krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:34.090109 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:34.430080 8265 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:34.430356 8265 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:34.430492 8265 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:34.437497 8265 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:34.437597 8265 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:34.437674 8265 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:34.437757 8265 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:34.437819 8265 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:34.447209 8265 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41109
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:34.448437 8265 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:34.449709 8265 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:34.460734 8277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:34.460706 8278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:34.460755 8284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:34.461531 8265 server_base.cc:1034] running on GCE node
I20260811 12:14:34.462292 8265 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:34.463644 8265 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:34.464890 8265 hybrid_clock.cc:648] HybridClock initialized: now 1786450474464848 us; error 40 us; skew 500 ppm
I20260811 12:14:34.465210 8265 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:34 dist-test-slave-nj21 krb5kdc[8243](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450474, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:34.468161 8265 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:34.469515 8265 webserver.cc:460] Webserver started at http://127.0.0.1:33639/ using document root <none> and password file <none>
I20260811 12:14:34.470155 8265 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:34.470270 8265 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:34.470557 8265 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:34.473491 8265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "51ef4d4dd3c9492fb8d8777ab2247541"
format_stamp: "Formatted at 2026-08-11 12:14:34 on dist-test-slave-nj21"
server_key: "2eeb171457ba285783a3469d3ff48539"
server_key_iv: "2e3e1a57f380c2454198f744777f928e"
server_key_version: "encryptionkey@0"
I20260811 12:14:34.474236 8265 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "51ef4d4dd3c9492fb8d8777ab2247541"
format_stamp: "Formatted at 2026-08-11 12:14:34 on dist-test-slave-nj21"
server_key: "2eeb171457ba285783a3469d3ff48539"
server_key_iv: "2e3e1a57f380c2454198f744777f928e"
server_key_version: "encryptionkey@0"
I20260811 12:14:34.479074 8265 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 12:14:34.482365 8293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:34.483623 8265 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:14:34.483793 8265 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "51ef4d4dd3c9492fb8d8777ab2247541"
format_stamp: "Formatted at 2026-08-11 12:14:34 on dist-test-slave-nj21"
server_key: "2eeb171457ba285783a3469d3ff48539"
server_key_iv: "2e3e1a57f380c2454198f744777f928e"
server_key_version: "encryptionkey@0"
I20260811 12:14:34.483960 8265 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:34.502326 8265 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:34.506364 8265 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:34.506595 8265 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:34.523152 8265 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41109
I20260811 12:14:34.523185 8380 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41109 every 8 connection(s)
I20260811 12:14:34.524499 8265 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:34.528194 8382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:34.534588 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8265
I20260811 12:14:34.534783 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:34.535389 4291 external_mini_cluster.cc:1428] Setting key 04c13d3e7d90027da9896cb715deaf13
I20260811 12:14:34.539268 8382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541: Bootstrap starting.
I20260811 12:14:34.542424 8382 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:34.543665 8382 log.cc:824] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:34.546861 8382 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541: No bootstrap required, opened a new log
I20260811 12:14:34.548834 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:34.537389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:54701 (local address 127.0.0.1:41109)
0811 12:14:34.538123 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:34.538134 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:34.538175 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:34.539905 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:34.539928 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:34.540293 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:34.540644 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:34.541196 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:34.542445 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:34.544244 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:34.545217 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:34.545351 (+ 134us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:34.546584 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:34.546592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:34.546606 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:34.547150 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:34.547221 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:34.547248 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:34.547372 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:34.547662 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 12:14:34.547873 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":535,"thread_start_us":313,"threads_started":1}
I20260811 12:14:34.552799 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:34.556167 8382 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 12:14:34.556648 8382 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:34.556766 8382 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ef4d4dd3c9492fb8d8777ab2247541, State: Initialized, Role: FOLLOWER
I20260811 12:14:34.557411 8382 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 12:14:34.557693 8382 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:34.557848 8382 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:34.558038 8382 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:34.561097 8382 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 12:14:34.561654 8382 leader_election.cc:302] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51ef4d4dd3c9492fb8d8777ab2247541; no voters:
I20260811 12:14:34.562340 8382 leader_election.cc:288] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:34.562443 8399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:34.564069 8399 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [term 1 LEADER]: Becoming Leader. State: Replica: 51ef4d4dd3c9492fb8d8777ab2247541, State: Running, Role: LEADER
I20260811 12:14:34.564435 8382 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:34.564657 8399 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 12:14:34.565546 8387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:34.557908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:50951 (local address 127.0.0.1:41109)
0811 12:14:34.558090 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:34.558096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:34.558582 (+ 486us) server_negotiation.cc:407] Connection header received
0811 12:14:34.558769 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:34.558772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:34.558839 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:34.558920 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:34.559466 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:34.560247 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:34.562077 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:34.562813 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:34.562949 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:34.564523 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:34.564530 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:34.564534 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:34.564556 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:34.564629 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:34.564653 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:34.564771 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:34.565165 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 12:14:34.565219 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:14:34.569490 8401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260811 12:14:34.569908 8401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:34.571450 8402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51ef4d4dd3c9492fb8d8777ab2247541. Latest consensus state: current_term: 1 leader_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51ef4d4dd3c9492fb8d8777ab2247541" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260811 12:14:34.571817 8402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:34.572537 8418 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:34.576890 8418 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:34.584714 8418 catalog_manager.cc:1347] Generated new cluster ID: 8bd88da85b4b4d4381a70d6df2de67d5
I20260811 12:14:34.584825 8418 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:34.602015 8418 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:34.603178 8418 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:34.609081 8418 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51ef4d4dd3c9492fb8d8777ab2247541: Generated new TSK 0
I20260811 12:14:34.609782 8418 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:34.618671 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8265
W20260811 12:14:34.619702 8373 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:54701
W20260811 12:14:34.620396 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8265: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:34.620543 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8265
W20260811 12:14:34.645563 8241 connection.cc:535] client connection to 127.0.0.1:41109 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41109): Connection reset by peer (error 104)
2026-08-11T12:14:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2064 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:34 dist-test-slave-nj21 krb5kdc[8431](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:34 dist-test-slave-nj21 krb5kdc[8431](info): set up 2 sockets
Aug 11 12:14:34 dist-test-slave-nj21 krb5kdc[8431](info): commencing operation
krb5kdc: starting...
W20260811 12:14:35.868145 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.204s user 0.004s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:36 dist-test-slave-nj21 krb5kdc[8431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 12:14:36.156131 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44755
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:36.484998 8457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:36.485265 8457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:36.485391 8457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:36.492453 8457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:36.492554 8457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:36.492627 8457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:36.492704 8457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:36.492769 8457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:36.502199 8457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44755
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:36.503436 8457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:36.504702 8457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:36.515790 8464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:36.515844 8465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:36.515866 8472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:36.516552 8457 server_base.cc:1034] running on GCE node
I20260811 12:14:36.517257 8457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:36.518519 8457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:36.519747 8457 hybrid_clock.cc:648] HybridClock initialized: now 1786450476519699 us; error 27 us; skew 500 ppm
I20260811 12:14:36.520056 8457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:36 dist-test-slave-nj21 krb5kdc[8431](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450476, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:36.527760 8457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 12:14:36.529150 8457 webserver.cc:460] Webserver started at http://127.0.0.1:35163/ using document root <none> and password file <none>
I20260811 12:14:36.529788 8457 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:36.529910 8457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:36.530195 8457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:36.533107 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "ebfbf3d6adfe47479e39af90b02dc020"
format_stamp: "Formatted at 2026-08-11 12:14:36 on dist-test-slave-nj21"
server_key: "ecbac3cf360ab4adea63165a79b567e8"
server_key_iv: "19545f86332e522ad6f46066ed6ce6af"
server_key_version: "encryptionkey@0"
I20260811 12:14:36.533836 8457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "ebfbf3d6adfe47479e39af90b02dc020"
format_stamp: "Formatted at 2026-08-11 12:14:36 on dist-test-slave-nj21"
server_key: "ecbac3cf360ab4adea63165a79b567e8"
server_key_iv: "19545f86332e522ad6f46066ed6ce6af"
server_key_version: "encryptionkey@0"
I20260811 12:14:36.538731 8457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260811 12:14:36.541985 8486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:36.543174 8457 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:14:36.543314 8457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "ebfbf3d6adfe47479e39af90b02dc020"
format_stamp: "Formatted at 2026-08-11 12:14:36 on dist-test-slave-nj21"
server_key: "ecbac3cf360ab4adea63165a79b567e8"
server_key_iv: "19545f86332e522ad6f46066ed6ce6af"
server_key_version: "encryptionkey@0"
I20260811 12:14:36.543480 8457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:36.548830 8457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:36.552429 8457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:36.552680 8457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:36.569298 8457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44755
I20260811 12:14:36.569353 8566 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44755 every 8 connection(s)
I20260811 12:14:36.570567 8457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:36.571480 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8457
I20260811 12:14:36.571633 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:36.572075 4291 external_mini_cluster.cc:1428] Setting key c690e9e51c209e87c0493c70539f4dc2
I20260811 12:14:36.574555 8567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:36.579084 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:36.573679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:60257 (local address 127.0.0.1:44755)
0811 12:14:36.574405 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:36.574416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:36.574457 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:36.576076 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:36.576097 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:36.576379 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:36.576672 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:36.577649 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:36.577655 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:36.577667 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:36.577687 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:36.577752 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:36.577773 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:36.577937 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:36.578050 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 12:14:36.578211 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":518,"thread_start_us":313,"threads_started":1}
I20260811 12:14:36.582521 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:36.585904 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: Bootstrap starting.
I20260811 12:14:36.589039 8571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:36.586778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.99:43171 (local address 127.0.0.1:44755)
0811 12:14:36.586954 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:36.586959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:36.587290 (+ 331us) server_negotiation.cc:407] Connection header received
0811 12:14:36.587451 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:36.587454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:36.587499 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:36.587575 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:36.588480 (+ 905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:36.588483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:36.588486 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:36.588500 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:36.588533 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:36.588547 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:36.588623 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:36.588747 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 12:14:36.588788 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:14:36.589550 8567 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:36.590546 8567 log.cc:824] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:36.593011 8567 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: No bootstrap required, opened a new log
I20260811 12:14:36.599817 8567 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } }
I20260811 12:14:36.600144 8567 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:36.600241 8567 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebfbf3d6adfe47479e39af90b02dc020, State: Initialized, Role: FOLLOWER
I20260811 12:14:36.600782 8567 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } }
I20260811 12:14:36.601001 8567 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:36.601126 8567 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:36.601272 8567 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:36.603247 8567 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } }
I20260811 12:14:36.603751 8567 leader_election.cc:302] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebfbf3d6adfe47479e39af90b02dc020; no voters:
I20260811 12:14:36.604282 8567 leader_election.cc:288] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:36.604380 8586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:36.605020 8586 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [term 1 LEADER]: Becoming Leader. State: Replica: ebfbf3d6adfe47479e39af90b02dc020, State: Running, Role: LEADER
I20260811 12:14:36.605515 8586 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } }
I20260811 12:14:36.606107 8567 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:36.608183 8588 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebfbf3d6adfe47479e39af90b02dc020. Latest consensus state: current_term: 1 leader_uuid: "ebfbf3d6adfe47479e39af90b02dc020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } } }
I20260811 12:14:36.608570 8588 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:36.613322 8587 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebfbf3d6adfe47479e39af90b02dc020" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebfbf3d6adfe47479e39af90b02dc020" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44755 } } }
I20260811 12:14:36.613610 8587 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020 [sys.catalog]: This master's current role is: LEADER
W20260811 12:14:36.618597 8614 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:14:36.618701 8614 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:14:36.618840 8593 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:36.620988 8593 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:36.629386 8593 catalog_manager.cc:1347] Generated new cluster ID: 8c7278d67a8c4029a1a2325fcb79df2e
I20260811 12:14:36.629495 8593 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:36.643098 8593 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:36.643508 8593 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:36.664870 8593 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebfbf3d6adfe47479e39af90b02dc020: Generated new TSK 0
I20260811 12:14:36.665757 8593 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:36.676244 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8457
W20260811 12:14:36.677670 8559 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.99:60257
W20260811 12:14:36.678332 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8457: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:36.678454 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8457
2026-08-11T12:14:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2055 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:36 dist-test-slave-nj21 krb5kdc[8626](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:36 dist-test-slave-nj21 krb5kdc[8626](info): set up 2 sockets
Aug 11 12:14:36 dist-test-slave-nj21 krb5kdc[8626](info): commencing operation
krb5kdc: starting...
W20260811 12:14:37.921734 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.203s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:38.194275 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36165
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:36165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:38.516742 8648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:38.517012 8648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:38.517138 8648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:38.524057 8648 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:38.524156 8648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:38.524235 8648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:38.524312 8648 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:38.524369 8648 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:38.533752 8648 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:36165
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:38.534958 8648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:38.536235 8648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:38.546984 8653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:38.547034 8662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:38.547058 8657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:38.548048 8648 server_base.cc:1034] running on GCE node
I20260811 12:14:38.548704 8648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:38.549950 8648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:38.551149 8648 hybrid_clock.cc:648] HybridClock initialized: now 1786450478551095 us; error 41 us; skew 500 ppm
I20260811 12:14:38.551471 8648 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8626](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450478, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:38.554369 8648 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:38.555707 8648 webserver.cc:460] Webserver started at http://127.4.48.254:40425/ using document root <none> and password file <none>
I20260811 12:14:38.556318 8648 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:38.556414 8648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:38.556644 8648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:38.559362 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "b2a5181b5e15415cba2aa5703b797d85"
format_stamp: "Formatted at 2026-08-11 12:14:38 on dist-test-slave-nj21"
server_key: "adfb8f7a23e7f959eebf3d85a6e4b33e"
server_key_iv: "63c3180d2a1dc28ad1dbc00dfc435169"
server_key_version: "encryptionkey@0"
I20260811 12:14:38.560092 8648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "b2a5181b5e15415cba2aa5703b797d85"
format_stamp: "Formatted at 2026-08-11 12:14:38 on dist-test-slave-nj21"
server_key: "adfb8f7a23e7f959eebf3d85a6e4b33e"
server_key_iv: "63c3180d2a1dc28ad1dbc00dfc435169"
server_key_version: "encryptionkey@0"
I20260811 12:14:38.564805 8648 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 12:14:38.568014 8674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:38.569202 8648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 12:14:38.569342 8648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "b2a5181b5e15415cba2aa5703b797d85"
format_stamp: "Formatted at 2026-08-11 12:14:38 on dist-test-slave-nj21"
server_key: "adfb8f7a23e7f959eebf3d85a6e4b33e"
server_key_iv: "63c3180d2a1dc28ad1dbc00dfc435169"
server_key_version: "encryptionkey@0"
I20260811 12:14:38.569507 8648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:38.594272 8648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:38.597882 8648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:38.598110 8648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:38.615177 8648 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:36165
I20260811 12:14:38.615208 8752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:36165 every 8 connection(s)
I20260811 12:14:38.616472 8648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:38.616770 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8648
I20260811 12:14:38.616919 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:38.617494 4291 external_mini_cluster.cc:1428] Setting key 87d1a55009cdd373c49517af8cce9914
I20260811 12:14:38.620916 8753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:38.631163 8753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85: Bootstrap starting.
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8626](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:38.634863 8753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:38.636209 8753 log.cc:824] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:38.639496 8753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85: No bootstrap required, opened a new log
I20260811 12:14:38.640897 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:38.619386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55053 (local address 127.4.48.254:36165)
0811 12:14:38.620243 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:38.620255 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:38.620302 (+ 47us) server_negotiation.cc:407] Connection header received
0811 12:14:38.622086 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:38.622112 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:38.622498 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:38.622876 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:38.624572 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:38.626013 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:38.627356 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:38.628103 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:38.628305 (+ 202us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:38.632548 (+ 4243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:38.632560 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:38.632582 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:38.632639 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:38.636440 (+ 3801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:38.637320 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:38.637327 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:38.637339 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:38.637509 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:38.637948 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:38.637953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:38.637956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:38.638623 (+ 667us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:38.638937 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:38.639258 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 12:14:38.639589 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":383,"threads_started":1}
I20260811 12:14:38.644650 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:38.649049 8753 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } }
I20260811 12:14:38.649508 8753 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:38.649618 8753 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2a5181b5e15415cba2aa5703b797d85, State: Initialized, Role: FOLLOWER
I20260811 12:14:38.650233 8753 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } }
I20260811 12:14:38.650518 8753 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:38.650671 8753 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:38.650867 8753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:38.653684 8753 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } }
I20260811 12:14:38.654258 8753 leader_election.cc:302] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2a5181b5e15415cba2aa5703b797d85; no voters:
I20260811 12:14:38.654911 8753 leader_election.cc:288] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:38.655011 8768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:38.657107 8753 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:38.662019 8768 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [term 1 LEADER]: Becoming Leader. State: Replica: b2a5181b5e15415cba2aa5703b797d85, State: Running, Role: LEADER
I20260811 12:14:38.662904 8768 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } }
I20260811 12:14:38.665700 8769 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2a5181b5e15415cba2aa5703b797d85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } } }
I20260811 12:14:38.666095 8769 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:38.667292 8757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:38.649743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39807 (local address 127.4.48.254:36165)
0811 12:14:38.649909 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:38.649914 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:38.650561 (+ 647us) server_negotiation.cc:407] Connection header received
0811 12:14:38.650768 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:38.650773 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:38.650851 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:38.650935 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:38.652676 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:38.653428 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:38.655117 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:38.655869 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:38.655962 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:38.659297 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:38.659306 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:38.659312 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:38.659358 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:38.663698 (+ 4340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:38.664387 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:38.664394 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:38.664398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:38.664655 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:38.665250 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:38.665257 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:38.665261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:38.665570 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:38.665906 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:38.666751 (+ 845us) server_negotiation.cc:299] Negotiation successful
0811 12:14:38.666951 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 12:14:38.668211 8770 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2a5181b5e15415cba2aa5703b797d85. Latest consensus state: current_term: 1 leader_uuid: "b2a5181b5e15415cba2aa5703b797d85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2a5181b5e15415cba2aa5703b797d85" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36165 } } }
I20260811 12:14:38.668565 8770 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:38.668926 8789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:38.673679 8789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:38.682633 8789 catalog_manager.cc:1347] Generated new cluster ID: f8ae37b7ab754d20afab776039659e1f
I20260811 12:14:38.682746 8789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:38.689921 8789 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:38.691162 8789 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:38.696260 8789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2a5181b5e15415cba2aa5703b797d85: Generated new TSK 0
I20260811 12:14:38.696979 8789 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:38.707647 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8648
I20260811 12:14:38.709161 8747 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:38.796188 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8648
W20260811 12:14:38.822551 8622 connection.cc:535] client connection to 127.4.48.254:36165 recv error: Network error: failed to read from TLS socket (remote: 127.4.48.254:36165): Connection reset by peer (error 104)
2026-08-11T12:14:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2121 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8811](info): set up 2 sockets
Aug 11 12:14:38 dist-test-slave-nj21 krb5kdc[8811](info): commencing operation
krb5kdc: starting...
W20260811 12:14:40.047564 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.207s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:40 dist-test-slave-nj21 krb5kdc[8811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:40.324010 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:32935
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:36441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:32935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:40.648932 8837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:40.649210 8837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:40.649348 8837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:40.656263 8837 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:40.656373 8837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:40.656452 8837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:40.656538 8837 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:40.656601 8837 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:40.665984 8837 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:36441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:32935
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:32935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:40.667201 8837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:40.668489 8837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:40.679613 8845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:40.679661 8847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:40.679710 8851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:40.680606 8837 server_base.cc:1034] running on GCE node
I20260811 12:14:40.681299 8837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:40.682554 8837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:40.683760 8837 hybrid_clock.cc:648] HybridClock initialized: now 1786450480683711 us; error 30 us; skew 500 ppm
I20260811 12:14:40.684128 8837 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:40 dist-test-slave-nj21 krb5kdc[8811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450480, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:40.686983 8837 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:40.688364 8837 webserver.cc:460] Webserver started at http://127.4.48.254:46425/ using document root <none> and password file <none>
I20260811 12:14:40.688997 8837 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:40.689112 8837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:40.689399 8837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:40.692267 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "7007e44792a4489c835aec9eac96495a"
format_stamp: "Formatted at 2026-08-11 12:14:40 on dist-test-slave-nj21"
server_key: "7b83a23a28a4d997badb078bc6952b11"
server_key_iv: "b743de9b0c472f7ef41a8d9123bbb59f"
server_key_version: "encryptionkey@0"
I20260811 12:14:40.693001 8837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "7007e44792a4489c835aec9eac96495a"
format_stamp: "Formatted at 2026-08-11 12:14:40 on dist-test-slave-nj21"
server_key: "7b83a23a28a4d997badb078bc6952b11"
server_key_iv: "b743de9b0c472f7ef41a8d9123bbb59f"
server_key_version: "encryptionkey@0"
I20260811 12:14:40.697844 8837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:14:40.701031 8859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:40.702199 8837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:14:40.702354 8837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "7007e44792a4489c835aec9eac96495a"
format_stamp: "Formatted at 2026-08-11 12:14:40 on dist-test-slave-nj21"
server_key: "7b83a23a28a4d997badb078bc6952b11"
server_key_iv: "b743de9b0c472f7ef41a8d9123bbb59f"
server_key_version: "encryptionkey@0"
I20260811 12:14:40.702521 8837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:40.715869 8837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:40.719460 8837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:40.719723 8837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:40.736459 8837 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:32935
I20260811 12:14:40.736495 8947 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:32935 every 8 connection(s)
I20260811 12:14:40.737739 8837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:40.741291 8948 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:40.747609 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 8837
I20260811 12:14:40.747816 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:40.748355 4291 external_mini_cluster.cc:1428] Setting key 51a98810028ef3bd90f12da1ecbf013b
I20260811 12:14:40.751876 8948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a: Bootstrap starting.
I20260811 12:14:40.755018 8948 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:40.756017 8948 log.cc:824] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:40.758628 8948 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a: No bootstrap required, opened a new log
I20260811 12:14:40.760598 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:40.750111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56995 (local address 127.4.48.254:32935)
0811 12:14:40.750852 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:40.750863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:40.750904 (+ 41us) server_negotiation.cc:407] Connection header received
0811 12:14:40.752570 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:40.752593 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:40.752942 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:40.753244 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:40.753829 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:40.755073 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:40.756425 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:40.757100 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:40.757262 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:40.758455 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:40.758462 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:40.758475 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:40.759007 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:40.759085 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:40.759109 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:40.759250 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:40.759530 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 12:14:40.759703 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":354,"threads_started":1}
I20260811 12:14:40.764093 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:40.766121 8948 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } }
I20260811 12:14:40.766445 8948 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:40.766541 8948 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7007e44792a4489c835aec9eac96495a, State: Initialized, Role: FOLLOWER
I20260811 12:14:40.767098 8948 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } }
I20260811 12:14:40.767315 8948 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:40.767438 8948 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:40.767596 8948 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:40.770325 8948 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } }
I20260811 12:14:40.770851 8948 leader_election.cc:302] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7007e44792a4489c835aec9eac96495a; no voters:
I20260811 12:14:40.771474 8948 leader_election.cc:288] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:40.771557 8967 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:40.773571 8948 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:40.775499 8954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:40.768286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34869 (local address 127.4.48.254:32935)
0811 12:14:40.768489 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:40.768494 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:40.768866 (+ 372us) server_negotiation.cc:407] Connection header received
0811 12:14:40.768985 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:40.768989 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:40.769045 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:40.769122 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:40.769661 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:40.770358 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:40.772223 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:40.772888 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:40.772984 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:40.774489 (+ 1505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:40.774496 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:40.774502 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:40.774524 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:40.774600 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:40.774622 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:40.774741 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:40.775056 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 12:14:40.775122 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 12:14:40.778126 8967 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [term 1 LEADER]: Becoming Leader. State: Replica: 7007e44792a4489c835aec9eac96495a, State: Running, Role: LEADER
I20260811 12:14:40.778592 8967 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } }
I20260811 12:14:40.780819 8968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7007e44792a4489c835aec9eac96495a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } } }
I20260811 12:14:40.781241 8968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:40.781778 8969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7007e44792a4489c835aec9eac96495a. Latest consensus state: current_term: 1 leader_uuid: "7007e44792a4489c835aec9eac96495a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7007e44792a4489c835aec9eac96495a" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 32935 } } }
I20260811 12:14:40.782102 8969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:40.783706 8982 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:40.788753 8982 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:40.796875 8982 catalog_manager.cc:1347] Generated new cluster ID: 622aebdaa30645d1b017e14d608edc77
I20260811 12:14:40.796980 8982 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:40.818089 8982 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:40.819902 8982 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:40.828457 8982 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7007e44792a4489c835aec9eac96495a: Generated new TSK 0
I20260811 12:14:40.829421 8982 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:40.838979 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8837
W20260811 12:14:40.840093 8941 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56995
W20260811 12:14:40.840770 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 8837: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:40.840906 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 8837
W20260811 12:14:40.865170 8802 connection.cc:535] client connection to 127.4.48.254:32935 recv error: Network error: failed to read from TLS socket (remote: 127.4.48.254:32935): Connection reset by peer (error 104)
2026-08-11T12:14:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2042 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:40 dist-test-slave-nj21 krb5kdc[8999](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:40 dist-test-slave-nj21 krb5kdc[8999](info): set up 2 sockets
Aug 11 12:14:40 dist-test-slave-nj21 krb5kdc[8999](info): commencing operation
krb5kdc: starting...
W20260811 12:14:42.068432 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.008s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:42 dist-test-slave-nj21 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:42.336772 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36453
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:36453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:42.657931 9021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:42.658195 9021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:42.658290 9021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:42.665164 9021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:42.665267 9021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:42.665320 9021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:42.665372 9021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:42.665422 9021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:42.674865 9021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:36453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:36453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:42.676075 9021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:42.677302 9021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:42.688305 9030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:42.688382 9032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:42.688401 9034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:42.689345 9021 server_base.cc:1034] running on GCE node
I20260811 12:14:42.690055 9021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:42.691311 9021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:42.692504 9021 hybrid_clock.cc:648] HybridClock initialized: now 1786450482692455 us; error 35 us; skew 500 ppm
I20260811 12:14:42.692832 9021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:42 dist-test-slave-nj21 krb5kdc[8999](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450482, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:42.698525 9021 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:42.699937 9021 webserver.cc:460] Webserver started at http://127.4.48.254:36013/ using document root <none> and password file <none>
I20260811 12:14:42.700584 9021 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:42.700702 9021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:42.700997 9021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:42.703972 9021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "1558cb2edecd4502804a2af4d0bc2ebc"
format_stamp: "Formatted at 2026-08-11 12:14:42 on dist-test-slave-nj21"
server_key: "afd04657ff111efceebce8e550013e76"
server_key_iv: "a14f8b7c508c53c6722314a7e5b9578d"
server_key_version: "encryptionkey@0"
I20260811 12:14:42.704680 9021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "1558cb2edecd4502804a2af4d0bc2ebc"
format_stamp: "Formatted at 2026-08-11 12:14:42 on dist-test-slave-nj21"
server_key: "afd04657ff111efceebce8e550013e76"
server_key_iv: "a14f8b7c508c53c6722314a7e5b9578d"
server_key_version: "encryptionkey@0"
I20260811 12:14:42.709589 9021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:14:42.712777 9047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:42.714066 9021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:42.714238 9021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "1558cb2edecd4502804a2af4d0bc2ebc"
format_stamp: "Formatted at 2026-08-11 12:14:42 on dist-test-slave-nj21"
server_key: "afd04657ff111efceebce8e550013e76"
server_key_iv: "a14f8b7c508c53c6722314a7e5b9578d"
server_key_version: "encryptionkey@0"
I20260811 12:14:42.714418 9021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:42.719894 9021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:42.723415 9021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:42.723697 9021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:42.740507 9021 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:36453
I20260811 12:14:42.740549 9143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:36453 every 8 connection(s)
I20260811 12:14:42.741811 9021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:42.745309 9145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:42.749181 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9021
I20260811 12:14:42.749358 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:42.749923 4291 external_mini_cluster.cc:1428] Setting key 85fa6c7dd53b34d6c496c2cf7a2b145c
I20260811 12:14:42.757452 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:42.751760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50225 (local address 127.4.48.254:36453)
0811 12:14:42.752561 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:42.752573 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:42.752617 (+ 44us) server_negotiation.cc:407] Connection header received
0811 12:14:42.754255 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:42.754278 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:42.754622 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:42.755053 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:42.756041 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:42.756047 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:42.756059 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:42.756073 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:42.756133 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:42.756153 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:42.756317 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:42.756418 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 12:14:42.756574 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":353,"threads_started":1}
I20260811 12:14:42.757933 9145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc: Bootstrap starting.
I20260811 12:14:42.760811 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:42.760998 9145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:42.762203 9145 log.cc:824] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:42.765307 9145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc: No bootstrap required, opened a new log
I20260811 12:14:42.768023 9149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:42.765496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57833 (local address 127.4.48.254:36453)
0811 12:14:42.765671 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:42.765677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:42.766156 (+ 479us) server_negotiation.cc:407] Connection header received
0811 12:14:42.766312 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:42.766316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:42.766369 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:42.766443 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:42.767376 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:42.767380 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:42.767383 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:42.767396 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:42.767436 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:42.767454 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:42.767543 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:42.767667 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 12:14:42.767746 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:14:42.773289 9145 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } }
I20260811 12:14:42.773625 9145 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:42.773728 9145 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1558cb2edecd4502804a2af4d0bc2ebc, State: Initialized, Role: FOLLOWER
I20260811 12:14:42.774315 9145 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } }
I20260811 12:14:42.774533 9145 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:42.774632 9145 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:42.774752 9145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:42.776844 9145 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } }
I20260811 12:14:42.777273 9145 leader_election.cc:302] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1558cb2edecd4502804a2af4d0bc2ebc; no voters:
I20260811 12:14:42.777787 9145 leader_election.cc:288] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:42.777918 9162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:42.778503 9162 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [term 1 LEADER]: Becoming Leader. State: Replica: 1558cb2edecd4502804a2af4d0bc2ebc, State: Running, Role: LEADER
I20260811 12:14:42.779024 9162 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } }
I20260811 12:14:42.779697 9145 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:42.781570 9164 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1558cb2edecd4502804a2af4d0bc2ebc. Latest consensus state: current_term: 1 leader_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } } }
I20260811 12:14:42.782092 9164 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:42.782052 9163 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1558cb2edecd4502804a2af4d0bc2ebc" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 36453 } } }
I20260811 12:14:42.782372 9163 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:42.784039 9172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:42.790848 9172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:42.799238 9172 catalog_manager.cc:1347] Generated new cluster ID: f51b9a7fec0c42ba9bbbb4308781727e
I20260811 12:14:42.799343 9172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:42.820887 9172 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:42.821353 9172 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:42.828653 9172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1558cb2edecd4502804a2af4d0bc2ebc: Generated new TSK 0
I20260811 12:14:42.829573 9172 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:42.839013 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9021
W20260811 12:14:42.840036 9138 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:50225
W20260811 12:14:42.840696 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9021: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:42.840847 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9021
2026-08-11T12:14:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2004 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:42 dist-test-slave-nj21 krb5kdc[9192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:42 dist-test-slave-nj21 krb5kdc[9192](info): set up 2 sockets
Aug 11 12:14:42 dist-test-slave-nj21 krb5kdc[9192](info): commencing operation
krb5kdc: starting...
W20260811 12:14:44.073553 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:44 dist-test-slave-nj21 krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:44.354732 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:40735
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:33025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:40735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:44.698400 9213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:44.698652 9213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:44.698747 9213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:44.705665 9213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:44.705762 9213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:44.705812 9213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:44.705870 9213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:44.705912 9213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:44.715237 9213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:33025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:40735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:40735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:44.716429 9213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:44.717621 9213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:44.728549 9223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:44.728466 9225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:44.728515 9230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:44.729432 9213 server_base.cc:1034] running on GCE node
I20260811 12:14:44.730170 9213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:44.731431 9213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:44.732622 9213 hybrid_clock.cc:648] HybridClock initialized: now 1786450484732577 us; error 37 us; skew 500 ppm
I20260811 12:14:44.732954 9213 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:44 dist-test-slave-nj21 krb5kdc[9192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450484, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:44.735841 9213 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:44.737186 9213 webserver.cc:460] Webserver started at http://127.4.48.254:42313/ using document root <none> and password file <none>
I20260811 12:14:44.737824 9213 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:44.737947 9213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:44.738232 9213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:44.741401 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "e012f00b76ab4d0dbbc40a3ece875f4d"
format_stamp: "Formatted at 2026-08-11 12:14:44 on dist-test-slave-nj21"
server_key: "def704848ab3556eea5f5bbdfd2a0aa1"
server_key_iv: "8687a520d77b713102596e201d7d3706"
server_key_version: "encryptionkey@0"
I20260811 12:14:44.742146 9213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "e012f00b76ab4d0dbbc40a3ece875f4d"
format_stamp: "Formatted at 2026-08-11 12:14:44 on dist-test-slave-nj21"
server_key: "def704848ab3556eea5f5bbdfd2a0aa1"
server_key_iv: "8687a520d77b713102596e201d7d3706"
server_key_version: "encryptionkey@0"
I20260811 12:14:44.746989 9213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 12:14:44.750262 9242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:44.751483 9213 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 12:14:44.751638 9213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "e012f00b76ab4d0dbbc40a3ece875f4d"
format_stamp: "Formatted at 2026-08-11 12:14:44 on dist-test-slave-nj21"
server_key: "def704848ab3556eea5f5bbdfd2a0aa1"
server_key_iv: "8687a520d77b713102596e201d7d3706"
server_key_version: "encryptionkey@0"
I20260811 12:14:44.751850 9213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:44.781103 9213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:44.784739 9213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:44.784989 9213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:44.801307 9213 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:40735
I20260811 12:14:44.801343 9327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:40735 every 8 connection(s)
I20260811 12:14:44.802592 9213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:44.806398 9329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:44.812057 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9213
I20260811 12:14:44.812248 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:44.812784 4291 external_mini_cluster.cc:1428] Setting key f4dd2eaea0997f44c0757197d700208b
I20260811 12:14:44.817155 9329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d: Bootstrap starting.
I20260811 12:14:44.820633 9329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:44.821821 9329 log.cc:824] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:44.825003 9329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d: No bootstrap required, opened a new log
Aug 11 12:14:44 dist-test-slave-nj21 krb5kdc[9192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:44.833472 9329 raft_consensus.cc:348] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } }
I20260811 12:14:44.833889 9329 raft_consensus.cc:374] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:44.834009 9329 raft_consensus.cc:729] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e012f00b76ab4d0dbbc40a3ece875f4d, State: Initialized, Role: FOLLOWER
I20260811 12:14:44.834672 9329 consensus_queue.cc:260] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } }
I20260811 12:14:44.834905 9329 raft_consensus.cc:388] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:44.835045 9329 raft_consensus.cc:482] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:44.835199 9329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:44.835608 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:44.814513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51429 (local address 127.4.48.254:40735)
0811 12:14:44.815336 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:44.815347 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:44.815387 (+ 40us) server_negotiation.cc:407] Connection header received
0811 12:14:44.817112 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:44.817152 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:44.817546 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:44.817852 (+ 306us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:44.819579 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:44.820905 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:44.822674 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:44.823522 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:44.823659 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:44.828546 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:44.828553 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:44.828569 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:44.828612 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:44.831633 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:44.832365 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:44.832371 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:44.832376 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:44.832549 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:44.832954 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:44.832957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:44.832959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:44.833446 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:44.833647 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:44.834163 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 12:14:44.834540 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":389,"threads_started":1}
I20260811 12:14:44.837893 9329 raft_consensus.cc:504] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } }
I20260811 12:14:44.838398 9329 leader_election.cc:302] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e012f00b76ab4d0dbbc40a3ece875f4d; no voters:
I20260811 12:14:44.838995 9329 leader_election.cc:288] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:44.839105 9337 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:44.839160 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:44.841125 9329 sys_catalog.cc:564] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:44.846021 9337 raft_consensus.cc:686] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [term 1 LEADER]: Becoming Leader. State: Replica: e012f00b76ab4d0dbbc40a3ece875f4d, State: Running, Role: LEADER
I20260811 12:14:44.846596 9337 consensus_queue.cc:237] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } }
I20260811 12:14:44.854882 9338 sys_catalog.cc:455] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } } }
I20260811 12:14:44.855649 9338 sys_catalog.cc:458] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:44.855357 9339 sys_catalog.cc:455] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e012f00b76ab4d0dbbc40a3ece875f4d. Latest consensus state: current_term: 1 leader_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e012f00b76ab4d0dbbc40a3ece875f4d" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40735 } } }
I20260811 12:14:44.856165 9339 sys_catalog.cc:458] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:44.860110 9368 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:44.863978 9333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:44.845782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43179 (local address 127.4.48.254:40735)
0811 12:14:44.845955 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:44.845961 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:44.848180 (+ 2219us) server_negotiation.cc:407] Connection header received
0811 12:14:44.848329 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:44.848334 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:44.848394 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:44.848470 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:44.849866 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:44.850629 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:44.851954 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:44.852383 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:44.852490 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:44.858005 (+ 5515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:44.858012 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:44.858020 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:44.858057 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:44.861332 (+ 3275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:44.862012 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:44.862017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:44.862020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:44.862148 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:44.862504 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:44.862507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:44.862509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:44.862846 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:44.863094 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:44.863427 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 12:14:44.863635 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:14:44.865231 9368 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:44.873649 9368 catalog_manager.cc:1347] Generated new cluster ID: e3dcd9e07b43495497deee02805c6270
I20260811 12:14:44.873739 9368 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:44.898464 9368 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:44.900256 9368 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:44.910802 9368 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e012f00b76ab4d0dbbc40a3ece875f4d: Generated new TSK 0
I20260811 12:14:44.911708 9368 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:44.918861 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9213
I20260811 12:14:44.920805 9322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:45.011042 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9213
2026-08-11T12:14:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:45 dist-test-slave-nj21 krb5kdc[9386](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:45 dist-test-slave-nj21 krb5kdc[9386](info): set up 2 sockets
Aug 11 12:14:45 dist-test-slave-nj21 krb5kdc[9386](info): commencing operation
krb5kdc: starting...
W20260811 12:14:46.250380 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:46 dist-test-slave-nj21 krb5kdc[9386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:46.517478 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46313
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:44761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:46313
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:46.842099 9407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:46.842370 9407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:46.842515 9407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:46.849485 9407 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:46.849607 9407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:46.849689 9407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:46.849792 9407 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:46.849852 9407 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:46.859289 9407 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:44761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:46313
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:46313
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:46.860476 9407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:46.861732 9407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:46.872668 9413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:46.872722 9417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:46.872727 9414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:46.873525 9407 server_base.cc:1034] running on GCE node
I20260811 12:14:46.874259 9407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:46.875519 9407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:46.876719 9407 hybrid_clock.cc:648] HybridClock initialized: now 1786450486876667 us; error 38 us; skew 500 ppm
I20260811 12:14:46.877043 9407 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:46 dist-test-slave-nj21 krb5kdc[9386](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450486, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:46.880045 9407 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:46.881366 9407 webserver.cc:460] Webserver started at http://127.4.48.254:42159/ using document root <none> and password file <none>
I20260811 12:14:46.882012 9407 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:46.882128 9407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:46.882439 9407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:46.885337 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "af95f47d78e2437eaf5eab27163c3866"
format_stamp: "Formatted at 2026-08-11 12:14:46 on dist-test-slave-nj21"
server_key: "a105ed77c6d49f1bebb60e24df4751ee"
server_key_iv: "8933f3e5529b58c35d6d5276004a05c2"
server_key_version: "encryptionkey@0"
I20260811 12:14:46.886086 9407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "af95f47d78e2437eaf5eab27163c3866"
format_stamp: "Formatted at 2026-08-11 12:14:46 on dist-test-slave-nj21"
server_key: "a105ed77c6d49f1bebb60e24df4751ee"
server_key_iv: "8933f3e5529b58c35d6d5276004a05c2"
server_key_version: "encryptionkey@0"
I20260811 12:14:46.890973 9407 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 12:14:46.894249 9428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:46.895418 9407 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:14:46.895567 9407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "af95f47d78e2437eaf5eab27163c3866"
format_stamp: "Formatted at 2026-08-11 12:14:46 on dist-test-slave-nj21"
server_key: "a105ed77c6d49f1bebb60e24df4751ee"
server_key_iv: "8933f3e5529b58c35d6d5276004a05c2"
server_key_version: "encryptionkey@0"
I20260811 12:14:46.895792 9407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:46.931874 9407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:46.935734 9407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:46.936002 9407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:46.954300 9407 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:46313
I20260811 12:14:46.954317 9519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:46313 every 8 connection(s)
I20260811 12:14:46.955715 9407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:46.959378 9523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:46.960839 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9407
I20260811 12:14:46.961011 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:46.961500 4291 external_mini_cluster.cc:1428] Setting key 8b2fc75decfeb531c19c240ef56d7bc4
I20260811 12:14:46.973315 9523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866: Bootstrap starting.
I20260811 12:14:46.975505 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:46.963423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36187 (local address 127.4.48.254:46313)
0811 12:14:46.964318 (+ 895us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:46.964337 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:46.964389 (+ 52us) server_negotiation.cc:407] Connection header received
0811 12:14:46.966025 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:46.966054 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:46.966552 (+ 498us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:46.966996 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:46.967454 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:46.969331 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:46.970617 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:46.971632 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:46.971817 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:46.972868 (+ 1051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:46.972879 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:46.972895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:46.973580 (+ 685us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:46.973678 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:46.973709 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:46.973821 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:46.974004 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 12:14:46.974249 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":394,"threads_started":1}
I20260811 12:14:46.977025 9523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:46.978113 9523 log.cc:824] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:46.978869 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:46.981140 9523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866: No bootstrap required, opened a new log
I20260811 12:14:46.989876 9527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:46.983764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37031 (local address 127.4.48.254:46313)
0811 12:14:46.983942 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:46.983948 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:46.984407 (+ 459us) server_negotiation.cc:407] Connection header received
0811 12:14:46.984521 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:46.984527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:46.984599 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:46.984705 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:46.985130 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:46.986188 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:46.987393 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:46.988104 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:46.988218 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:46.989162 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:46.989168 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:46.989173 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:46.989192 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:46.989239 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:46.989259 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:46.989375 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:46.989460 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 12:14:46.989546 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:14:46.990708 9523 raft_consensus.cc:348] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } }
I20260811 12:14:46.991174 9523 raft_consensus.cc:374] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:46.991286 9523 raft_consensus.cc:729] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af95f47d78e2437eaf5eab27163c3866, State: Initialized, Role: FOLLOWER
I20260811 12:14:46.991860 9523 consensus_queue.cc:260] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } }
I20260811 12:14:46.992076 9523 raft_consensus.cc:388] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:46.992172 9523 raft_consensus.cc:482] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:46.992292 9523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:46.994416 9523 raft_consensus.cc:504] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } }
I20260811 12:14:46.994974 9523 leader_election.cc:302] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: af95f47d78e2437eaf5eab27163c3866; no voters:
I20260811 12:14:46.995604 9523 leader_election.cc:288] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:46.995714 9542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:46.996398 9542 raft_consensus.cc:686] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [term 1 LEADER]: Becoming Leader. State: Replica: af95f47d78e2437eaf5eab27163c3866, State: Running, Role: LEADER
I20260811 12:14:46.996954 9542 consensus_queue.cc:237] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } }
I20260811 12:14:46.997634 9523 sys_catalog.cc:564] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:46.999778 9545 sys_catalog.cc:455] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader af95f47d78e2437eaf5eab27163c3866. Latest consensus state: current_term: 1 leader_uuid: "af95f47d78e2437eaf5eab27163c3866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } } }
I20260811 12:14:47.000232 9545 sys_catalog.cc:458] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:47.000205 9544 sys_catalog.cc:455] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "af95f47d78e2437eaf5eab27163c3866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "af95f47d78e2437eaf5eab27163c3866" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 46313 } } }
I20260811 12:14:47.000506 9544 sys_catalog.cc:458] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:47.004696 9556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:47.009743 9556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:47.018064 9556 catalog_manager.cc:1347] Generated new cluster ID: d60e16f7100e4353a0e5034eaa758290
I20260811 12:14:47.018179 9556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:47.032814 9556 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:47.034578 9556 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:47.042353 9556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P af95f47d78e2437eaf5eab27163c3866: Generated new TSK 0
I20260811 12:14:47.043291 9556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:47.060158 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9407
W20260811 12:14:47.061767 9510 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36187
W20260811 12:14:47.062467 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9407: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:47.062589 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9407
2026-08-11T12:14:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2053 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:47 dist-test-slave-nj21 krb5kdc[9577](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:47 dist-test-slave-nj21 krb5kdc[9577](info): set up 2 sockets
Aug 11 12:14:47 dist-test-slave-nj21 krb5kdc[9577](info): commencing operation
krb5kdc: starting...
W20260811 12:14:48.304108 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.199s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:48 dist-test-slave-nj21 krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:48.590052 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39893
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:37579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:39893
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:48.924479 9601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:48.924785 9601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:48.924940 9601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:48.932092 9601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:48.932206 9601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:48.932288 9601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:48.932374 9601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:48.932437 9601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:48.942085 9601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:37579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:39893
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:39893
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:48.943513 9601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:48.944834 9601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:48.956122 9610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:48.956773 9608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:48.957306 9601 server_base.cc:1034] running on GCE node
W20260811 12:14:48.957768 9613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:48.958570 9601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:48.959965 9601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:48.961151 9601 hybrid_clock.cc:648] HybridClock initialized: now 1786450488961101 us; error 48 us; skew 500 ppm
I20260811 12:14:48.961472 9601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:48 dist-test-slave-nj21 krb5kdc[9577](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450488, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:48.967334 9601 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:48.968763 9601 webserver.cc:460] Webserver started at http://127.4.48.254:40793/ using document root <none> and password file <none>
I20260811 12:14:48.969410 9601 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:48.969539 9601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:48.969823 9601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:48.972724 9601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "41eccc68b3684f7e9ec4b995f0190418"
format_stamp: "Formatted at 2026-08-11 12:14:48 on dist-test-slave-nj21"
server_key: "18340f41f25b1df734f2cb25221f211c"
server_key_iv: "b31787f0f824fc7f9cfacc41fc19d42b"
server_key_version: "encryptionkey@0"
I20260811 12:14:48.973452 9601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "41eccc68b3684f7e9ec4b995f0190418"
format_stamp: "Formatted at 2026-08-11 12:14:48 on dist-test-slave-nj21"
server_key: "18340f41f25b1df734f2cb25221f211c"
server_key_iv: "b31787f0f824fc7f9cfacc41fc19d42b"
server_key_version: "encryptionkey@0"
I20260811 12:14:48.978384 9601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 12:14:48.981696 9637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:48.982885 9601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:14:48.983043 9601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "41eccc68b3684f7e9ec4b995f0190418"
format_stamp: "Formatted at 2026-08-11 12:14:48 on dist-test-slave-nj21"
server_key: "18340f41f25b1df734f2cb25221f211c"
server_key_iv: "b31787f0f824fc7f9cfacc41fc19d42b"
server_key_version: "encryptionkey@0"
I20260811 12:14:48.983220 9601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:48.988693 9601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:48.992193 9601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:48.992458 9601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:49.009620 9601 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:39893
I20260811 12:14:49.009667 9731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:39893 every 8 connection(s)
I20260811 12:14:49.010918 9601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:49.014642 9734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:49.016619 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9601
I20260811 12:14:49.016811 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:49.017303 4291 external_mini_cluster.cc:1428] Setting key 321e256bd87137dd1ed8e10f08350b36
I20260811 12:14:49.024791 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:49.018968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43549 (local address 127.4.48.254:39893)
0811 12:14:49.019803 (+ 835us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:49.019817 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:49.019870 (+ 53us) server_negotiation.cc:407] Connection header received
0811 12:14:49.021547 (+ 1677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:49.021569 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:49.021888 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:49.022176 (+ 288us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:49.023213 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:49.023219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:49.023235 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:49.023249 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:49.023309 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:49.023333 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:49.023505 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:49.023654 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 12:14:49.023909 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":364,"threads_started":1}
I20260811 12:14:49.026269 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418: Bootstrap starting.
I20260811 12:14:49.028112 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:49.029692 9734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:49.030944 9734 log.cc:824] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:49.034139 9734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418: No bootstrap required, opened a new log
I20260811 12:14:49.035799 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:49.033643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45711 (local address 127.4.48.254:39893)
0811 12:14:49.033788 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:49.033793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:49.034026 (+ 233us) server_negotiation.cc:407] Connection header received
0811 12:14:49.034193 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:49.034198 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:49.034251 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:49.034323 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:49.035197 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:49.035201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:49.035204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 12:14:49.035217 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:49.035251 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 12:14:49.035265 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:49.035340 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:49.035486 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:14:49.035528 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 12:14:49.041571 9734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } }
I20260811 12:14:49.041888 9734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 12:14:49.041985 9734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41eccc68b3684f7e9ec4b995f0190418, State: Initialized, Role: FOLLOWER
I20260811 12:14:49.042570 9734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } }
I20260811 12:14:49.042831 9734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 12:14:49.042974 9734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 12:14:49.043133 9734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:49.045208 9734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } }
I20260811 12:14:49.045679 9734 leader_election.cc:302] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41eccc68b3684f7e9ec4b995f0190418; no voters:
I20260811 12:14:49.046238 9734 leader_election.cc:288] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 12:14:49.046402 9748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:49.047432 9748 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [term 1 LEADER]: Becoming Leader. State: Replica: 41eccc68b3684f7e9ec4b995f0190418, State: Running, Role: LEADER
I20260811 12:14:49.047932 9748 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } }
I20260811 12:14:49.048316 9734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:49.050637 9750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41eccc68b3684f7e9ec4b995f0190418. Latest consensus state: current_term: 1 leader_uuid: "41eccc68b3684f7e9ec4b995f0190418" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } } }
I20260811 12:14:49.051079 9750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:49.051431 9749 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41eccc68b3684f7e9ec4b995f0190418" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41eccc68b3684f7e9ec4b995f0190418" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 39893 } } }
I20260811 12:14:49.051831 9749 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418 [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:49.053822 9763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:49.058626 9763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:49.066008 9763 catalog_manager.cc:1347] Generated new cluster ID: f95fcbd428a44d84a4ddb8a6aa8aba09
I20260811 12:14:49.066123 9763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:49.081593 9763 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:49.082062 9763 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:49.093155 9763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41eccc68b3684f7e9ec4b995f0190418: Generated new TSK 0
I20260811 12:14:49.094082 9763 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:49.102195 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9601
W20260811 12:14:49.103381 9723 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43549
W20260811 12:14:49.104058 4291 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9601: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 12:14:49.104175 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9601
2026-08-11T12:14:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2047 ms)
[----------] 18 tests from AuthTokenIssuingTest (37502 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:49 dist-test-slave-nj21 krb5kdc[9787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:49 dist-test-slave-nj21 krb5kdc[9787](info): set up 2 sockets
Aug 11 12:14:49 dist-test-slave-nj21 krb5kdc[9787](info): commencing operation
krb5kdc: starting...
W20260811 12:14:50.352185 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.198s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:50 dist-test-slave-nj21 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:50.644454 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:40159
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:50.976965 9811 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:50.977268 9811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:50.977344 9811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:50.977423 9811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:50.984380 9811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:50.984455 9811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:50.984519 9811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:50.984570 9811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:50.984637 9811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:50.993930 9811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:50.995074 9811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:50.996345 9811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:51.007239 9817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:51.007273 9822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:51.007202 9818 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:51.008247 9811 server_base.cc:1034] running on GCE node
I20260811 12:14:51.008932 9811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:51.010172 9811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:51.011400 9811 hybrid_clock.cc:648] HybridClock initialized: now 1786450491011334 us; error 52 us; skew 500 ppm
I20260811 12:14:51.011734 9811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:51.014708 9811 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:51.016074 9811 webserver.cc:460] Webserver started at http://127.4.48.254:38931/ using document root <none> and password file <none>
I20260811 12:14:51.016709 9811 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:51.016809 9811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:51.017104 9811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:51.020008 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "460f370c1f844bf68aaefdd1706c866f"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "d5035b9b6a7320dcf6d85637de115159"
server_key_iv: "ab6998b29c407229657ed78d12cf2818"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.020841 9811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "460f370c1f844bf68aaefdd1706c866f"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "d5035b9b6a7320dcf6d85637de115159"
server_key_iv: "ab6998b29c407229657ed78d12cf2818"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.025730 9811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 12:14:51.029057 9830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:51.030262 9811 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:14:51.030422 9811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "460f370c1f844bf68aaefdd1706c866f"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "d5035b9b6a7320dcf6d85637de115159"
server_key_iv: "ab6998b29c407229657ed78d12cf2818"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.030611 9811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:51.051189 9811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:51.055225 9811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:51.055478 9811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:51.072491 9811 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:40159
I20260811 12:14:51.072521 9920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:40159 every 8 connection(s)
I20260811 12:14:51.073737 9811 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:51.077183 9921 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:51.083262 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9811
I20260811 12:14:51.083469 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:51.084132 4291 external_mini_cluster.cc:1428] Setting key ff2971b140590af6dcf27c1df43b7b73
I20260811 12:14:51.083470 9921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:51.105415 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.086317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:40159 (local address 127.0.0.1:56112)
0811 12:14:51.087098 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.087157 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:51.089309 (+ 2152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:51.090111 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:51.090118 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:51.090807 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:51.091272 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.091296 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.092810 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.092815 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.094293 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.094307 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.095174 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.095180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.095363 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.096006 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:51.096034 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:51.098318 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:51.101860 (+ 3542us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.101869 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.101877 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.102250 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.102663 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.102666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.102668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.102806 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.103651 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:51.103661 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:51.103880 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 12:14:51.104089 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 12:14:51.104435 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":298,"threads_started":1}
I20260811 12:14:51.105440 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.087309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.4.48.254:40159)
0811 12:14:51.087884 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:51.087890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:51.087940 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:14:51.089681 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:51.089686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:51.089885 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:51.090037 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:51.091473 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.092658 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.094414 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.094944 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.095061 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.098547 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:51.098558 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:51.098573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:51.098607 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:51.101745 (+ 3138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.102378 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.102386 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.102389 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.102567 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.102903 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.102907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.102910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.103537 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:51.103781 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:51.104241 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 12:14:51.104595 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":91,"server-negotiator.queue_time_us":421,"thread_start_us":269,"threads_started":1}
I20260811 12:14:51.108567 9921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:51.109589 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.109172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40476)
0811 12:14:51.109330 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.109466 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:14:51.109810 9836 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
I20260811 12:14:51.111403 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.110978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40480)
0811 12:14:51.111141 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.111234 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 12:14:51.111603 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:43113: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.4.48.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
I20260811 12:14:51.154394 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:43113
--webserver_interface=127.4.48.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:14:51.160991 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } attempt: 1
I20260811 12:14:51.162050 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.161618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40490)
0811 12:14:51.161808 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.161894 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:14:51.162971 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.162625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40506)
0811 12:14:51.162764 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.162831 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:14:51.163138 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:43113: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
I20260811 12:14:51.231385 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } attempt: 2
I20260811 12:14:51.232677 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.232113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40514)
0811 12:14:51.232375 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.232492 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:14:51.233732 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.233303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40522)
0811 12:14:51.233452 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.233555 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:14:51.233916 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:43113: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
I20260811 12:14:51.325188 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } attempt: 3
I20260811 12:14:51.326439 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.325924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40536)
0811 12:14:51.326154 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.326269 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 12:14:51.327499 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.327116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40546)
0811 12:14:51.327258 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.327339 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:14:51.327745 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:43113: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
I20260811 12:14:51.471001 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } attempt: 4
I20260811 12:14:51.472221 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.471658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40562)
0811 12:14:51.472002 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.472090 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168}
I20260811 12:14:51.473152 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.472843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40570)
0811 12:14:51.472956 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.473035 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:14:51.473331 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:43113: Network error: Client connection negotiation failed: client connection to 127.4.48.253:43113: connect: Connection refused (error 111)
W20260811 12:14:51.483314 9931 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:51.483619 9931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:51.483727 9931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:51.483836 9931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:51.490994 9931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:51.491086 9931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:51.491178 9931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:51.491238 9931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:51.491323 9931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:51.501127 9931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:43113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.48.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:51.502410 9931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:51.503747 9931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:51.514868 9938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:51.514796 9939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:51.514832 9944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:51.515944 9931 server_base.cc:1034] running on GCE node
I20260811 12:14:51.516642 9931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:51.517882 9931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:51.519096 9931 hybrid_clock.cc:648] HybridClock initialized: now 1786450491519024 us; error 49 us; skew 500 ppm
I20260811 12:14:51.519402 9931 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:51.522416 9931 init.cc:377] Logged in from keytab as kudu/127.4.48.253@KRBTEST.COM (short username kudu)
I20260811 12:14:51.523747 9931 webserver.cc:460] Webserver started at http://127.4.48.253:34347/ using document root <none> and password file <none>
I20260811 12:14:51.524402 9931 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:51.524501 9931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:51.524806 9931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:51.527627 9931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data/instance:
uuid: "19781debdc9c4d9db76f86cb40ff2498"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "3407ffb6ae71d31ab175bd2e4ce2c2ab"
server_key_iv: "ef63d68e6550830b5d0e0e996f3d71a9"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.528471 9931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal/instance:
uuid: "19781debdc9c4d9db76f86cb40ff2498"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "3407ffb6ae71d31ab175bd2e4ce2c2ab"
server_key_iv: "ef63d68e6550830b5d0e0e996f3d71a9"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.533254 9931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:14:51.536564 9953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:51.537773 9931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:14:51.537923 9931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "19781debdc9c4d9db76f86cb40ff2498"
format_stamp: "Formatted at 2026-08-11 12:14:51 on dist-test-slave-nj21"
server_key: "3407ffb6ae71d31ab175bd2e4ce2c2ab"
server_key_iv: "ef63d68e6550830b5d0e0e996f3d71a9"
server_key_version: "encryptionkey@0"
I20260811 12:14:51.538105 9931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:51.552649 9931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:51.556596 9931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:51.556830 9931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:51.574011 9931 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.253:43113
I20260811 12:14:51.574033 10049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.253:43113 every 8 connection(s)
I20260811 12:14:51.575310 9931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
I20260811 12:14:51.578877 10052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:51.582075 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 9931
I20260811 12:14:51.582273 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-1/wal/instance
I20260811 12:14:51.582794 4291 external_mini_cluster.cc:1428] Setting key 1e2dd59c845bf9309b5f970466c8e881
I20260811 12:14:51.586724 10052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:51.607182 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.589711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.4.48.254:40159)
0811 12:14:51.589925 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:51.589931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:51.590975 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 12:14:51.592823 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:51.592835 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:51.592919 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:51.593025 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:51.594388 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.595274 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.596746 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.597187 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.597282 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.601183 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:51.601188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:51.601191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:51.601226 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:51.604211 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.604921 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.604926 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.604929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.605073 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.605462 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.605467 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.605470 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.605830 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:51.605977 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:51.606634 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 12:14:51.606869 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 12:14:51.607926 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.590037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:40159 (local address 127.0.0.1:56124)
0811 12:14:51.590802 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.590855 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:51.592571 (+ 1716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:51.593197 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:51.593204 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:51.593693 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:51.594219 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.594234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.595412 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.595416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.596602 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.596614 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.597422 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.597429 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.597700 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.598375 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:51.598403 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:51.600972 (+ 2569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:51.604354 (+ 3382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.604360 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.604372 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.604783 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.605161 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.605164 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.605166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.605337 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.605938 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:51.605944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:51.606238 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 12:14:51.606468 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 12:14:51.606828 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":343,"threads_started":1}
I20260811 12:14:51.609730 10052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.4.48.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.252@KRBTEST.COM" created.
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:51.626135 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.610336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40574)
0811 12:14:51.610514 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.610532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:51.610626 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:51.611815 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:51.611821 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:51.612227 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:51.612375 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.612384 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.614352 (+ 1968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.614356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.615263 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.615275 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.616432 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.616436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.616570 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.617119 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:51.617137 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:51.620264 (+ 3127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:51.623868 (+ 3604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.623871 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.623874 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.624196 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.624562 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.624565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.624567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.624700 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.625341 (+ 641us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:51.625348 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:51.625483 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 12:14:51.625600 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 12:14:51.625826 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:14:51.626398 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.610585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40574 (local address 127.4.48.253:43113)
0811 12:14:51.611312 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:51.611319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:51.611348 (+ 29us) server_negotiation.cc:407] Connection header received
0811 12:14:51.611396 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:51.611403 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:51.611608 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:51.611793 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:51.612570 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.614186 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.615415 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.616275 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.616412 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.620497 (+ 4085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:51.620510 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:51.620528 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:51.620564 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:51.623753 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.624313 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.624322 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.624333 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.624463 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.624790 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.624794 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.624796 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.625201 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:51.625409 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:51.625769 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 12:14:51.626006 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":386,"threads_started":1}
I20260811 12:14:51.628973 10052 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:51.630060 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.629591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56200)
0811 12:14:51.629759 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.629900 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 12:14:51.630321 9958 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.633932 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.633558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56204)
0811 12:14:51.633702 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.633794 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:14:51.634184 10052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
I20260811 12:14:51.654742 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:35813
--webserver_interface=127.4.48.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:42057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:14:51.683462 10052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 1
I20260811 12:14:51.684613 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.684175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56216)
0811 12:14:51.684359 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.684461 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:14:51.685551 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.685205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56226)
0811 12:14:51.685366 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.685443 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:14:51.685737 10052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.753928 10052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 2
I20260811 12:14:51.755048 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.754502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56230)
0811 12:14:51.754765 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.754883 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 12:14:51.756009 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.755640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56246)
0811 12:14:51.755810 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.755885 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 12:14:51.756203 10052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.772533 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } attempt: 5
Aug 11 12:14:51 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:51.785903 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.773149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40586)
0811 12:14:51.773338 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.773359 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:51.773468 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:51.773825 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:51.773828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:51.774188 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:51.774361 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.774370 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.775722 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.775725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.776556 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:51.776568 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.777561 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.777565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:51.777709 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.778264 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:51.778292 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:51.780314 (+ 2022us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:51.783853 (+ 3539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.783857 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.783860 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.784208 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.784560 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:51.784563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:51.784566 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:51.784685 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:51.785231 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:51.785234 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:51.785364 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 12:14:51.785467 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 12:14:51.785650 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 12:14:51.786113 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.773268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40586 (local address 127.4.48.253:43113)
0811 12:14:51.773435 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:51.773441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:51.773465 (+ 24us) server_negotiation.cc:407] Connection header received
0811 12:14:51.773585 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:51.773591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:51.773690 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:51.773794 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:51.774504 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.775574 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.776698 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:51.777392 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:51.777515 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:51.780512 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:51.780519 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:51.780524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:51.780563 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:51.783737 (+ 3174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.784313 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.784318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.784322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.784471 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:51.784773 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:51.784778 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:51.784781 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:51.785137 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:51.785280 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:51.785584 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 12:14:51.785801 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:14:51.787062 9921 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:51.788074 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.787591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56262)
0811 12:14:51.787783 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.787880 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 12:14:51.789016 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.788636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56278)
0811 12:14:51.788783 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.788885 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:14:51.789207 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.840417 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 1
I20260811 12:14:51.841614 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.841108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56288)
0811 12:14:51.841323 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.841429 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:14:51.842579 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.842265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56292)
0811 12:14:51.842385 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.842469 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 12:14:51.842765 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.847395 10052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 3
I20260811 12:14:51.848278 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.847874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56294)
0811 12:14:51.848062 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.848142 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 12:14:51.849234 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.848840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56310)
0811 12:14:51.849047 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.849119 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
W20260811 12:14:51.849450 10052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:51.910981 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 2
I20260811 12:14:51.912184 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.911645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56316)
0811 12:14:51.911918 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.912030 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 12:14:51.913134 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.912803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56328)
0811 12:14:51.912958 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.913029 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 12:14:51.913331 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
W20260811 12:14:51.985563 10064 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:51.985893 10064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:51.985992 10064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:51.986107 10064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 12:14:51.992635 10052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 4
W20260811 12:14:51.993136 10064 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:51.993216 10064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:51.993314 10064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:51.993391 10064 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:51.993475 10064 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:51.993806 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.993194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56330)
0811 12:14:51.993499 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.993611 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:14:51.994753 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:51.994407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56332)
0811 12:14:51.994540 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:51.994625 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 12:14:51.994958 10052 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
I20260811 12:14:52.003372 10064 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:42057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:40159,127.4.48.253:43113,127.4.48.252:35813
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:35813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.48.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:52.004585 10064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:52.005810 10064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:52.016835 10073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:52.017022 10076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:52.016865 10072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:52.017740 10064 server_base.cc:1034] running on GCE node
I20260811 12:14:52.018472 10064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:52.019536 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 3
I20260811 12:14:52.019778 10064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:52.020666 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.020123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56346)
0811 12:14:52.020377 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.020480 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:14:52.021023 10064 hybrid_clock.cc:648] HybridClock initialized: now 1786450492020946 us; error 44 us; skew 500 ppm
I20260811 12:14:52.021355 10064 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 12:14:52.021600 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.021230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56354)
0811 12:14:52.021406 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.021478 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 12:14:52.021800 9921 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:35813: Network error: Client connection negotiation failed: client connection to 127.4.48.252:35813: connect: Connection refused (error 111)
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:52.024477 10064 init.cc:377] Logged in from keytab as kudu/127.4.48.252@KRBTEST.COM (short username kudu)
I20260811 12:14:52.025840 10064 webserver.cc:460] Webserver started at http://127.4.48.252:35757/ using document root <none> and password file <none>
I20260811 12:14:52.026469 10064 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:52.026571 10064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:52.026875 10064 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:52.029852 10064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data/instance:
uuid: "6f697fcc60bc439a9aa2925521fee89e"
format_stamp: "Formatted at 2026-08-11 12:14:52 on dist-test-slave-nj21"
server_key: "21caac1329ce55c70ecca4384e490e5a"
server_key_iv: "316b27402be74093705584b40e61f889"
server_key_version: "encryptionkey@0"
I20260811 12:14:52.030583 10064 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal/instance:
uuid: "6f697fcc60bc439a9aa2925521fee89e"
format_stamp: "Formatted at 2026-08-11 12:14:52 on dist-test-slave-nj21"
server_key: "21caac1329ce55c70ecca4384e490e5a"
server_key_iv: "316b27402be74093705584b40e61f889"
server_key_version: "encryptionkey@0"
I20260811 12:14:52.035553 10064 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.006s
I20260811 12:14:52.038726 10087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:52.040099 10064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:52.040258 10064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "6f697fcc60bc439a9aa2925521fee89e"
format_stamp: "Formatted at 2026-08-11 12:14:52 on dist-test-slave-nj21"
server_key: "21caac1329ce55c70ecca4384e490e5a"
server_key_iv: "316b27402be74093705584b40e61f889"
server_key_version: "encryptionkey@0"
I20260811 12:14:52.040452 10064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:52.050309 10064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:52.054064 10064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:52.054288 10064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:52.071367 10064 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.252:35813
I20260811 12:14:52.071382 10178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.252:35813 every 8 connection(s)
I20260811 12:14:52.072979 10064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
I20260811 12:14:52.076691 10179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:52.080646 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10064
I20260811 12:14:52.080873 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786450132843697-4291-0/minicluster-data/master-2/wal/instance
I20260811 12:14:52.081419 4291 external_mini_cluster.cc:1428] Setting key 0be0863903e47fed24e68e1264632470
I20260811 12:14:52.084002 10179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:52.102969 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.087842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.4.48.254:40159)
0811 12:14:52.088053 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.088060 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.088637 (+ 577us) server_negotiation.cc:407] Connection header received
0811 12:14:52.088822 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.088827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.088889 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.088970 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.090183 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.090849 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.092404 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.092962 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.093055 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.096935 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.096942 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.096947 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.096986 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.100004 (+ 3018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.100828 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.100833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.100837 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.100992 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.101390 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.101395 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.101397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.101722 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.101869 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.102399 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.102615 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:14:52.103878 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.083168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56368 (local address 127.4.48.252:35813)
0811 12:14:52.083902 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.083912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.083951 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:14:52.085867 (+ 1916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.085889 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.086251 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.086537 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.088297 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.089803 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.091191 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.091876 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.092002 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.096589 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.096599 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.096609 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.096638 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.099955 (+ 3317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.100721 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.100727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.100729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.100935 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.101360 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.101364 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.101367 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.101970 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.102200 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.102594 (+ 394us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.102854 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":518,"thread_start_us":307,"threads_started":1}
I20260811 12:14:52.103916 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.087830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:40159 (local address 127.0.0.1:56134)
0811 12:14:52.088507 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.088550 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:52.088693 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:52.089048 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:52.089054 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:52.089670 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:52.090029 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.090043 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.091006 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.091011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.092269 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.092284 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.093180 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.093188 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.093377 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.093916 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:52.093945 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:52.096751 (+ 2806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:52.100135 (+ 3384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.100146 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.100157 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.100691 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.101093 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.101098 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.101101 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.101295 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.101830 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:52.101835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:52.102004 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 12:14:52.102285 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:14:52.102731 (+ 446us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"mutex_wait_us":28,"thread_start_us":328,"threads_started":1}
I20260811 12:14:52.105373 10179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:52.118631 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.105947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:43113 (local address 127.0.0.1:40588)
0811 12:14:52.106147 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.106162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:52.106263 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:52.106545 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:52.106550 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:52.106923 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:52.107080 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.107088 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.108123 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.108129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.109434 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.109462 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.110390 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.110394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.110540 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.111002 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:52.111018 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:52.113208 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:52.116475 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.116478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.116481 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.116817 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.117160 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.117164 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.117166 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.117302 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.117779 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:52.117784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:52.117926 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 12:14:52.118052 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 12:14:52.118309 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:14:52.118733 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.106103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40588 (local address 127.4.48.253:43113)
0811 12:14:52.106281 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.106285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.106305 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:14:52.106353 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.106357 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.106413 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.106487 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.107207 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.107974 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.109619 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.110130 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.110205 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.113396 (+ 3191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.113401 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.113404 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.113438 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.116335 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.116928 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.116932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.116934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.117057 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.117402 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.117407 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.117410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.117659 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.117795 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.118175 (+ 380us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.118384 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:14:52.119567 10179 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:52.120487 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.106930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56136 (local address 127.4.48.254:40159)
0811 12:14:52.107097 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.107103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.107118 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:14:52.107160 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.107163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.107222 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.107322 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.108483 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.109459 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.110793 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.111179 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.111342 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.115369 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.115377 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.115382 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.115419 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.117991 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.118580 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.118585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.118587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.118713 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.119253 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.119259 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.119262 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.119553 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.119866 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.120078 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.120262 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450492, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:52.132642 10186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.120276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56376)
0811 12:14:52.120512 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.120529 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:52.120729 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:52.121024 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:52.121027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:52.121435 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:52.121599 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.121611 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.122777 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.122782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.123723 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.123737 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.124503 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.124509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.124726 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.125231 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:52.125246 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:52.127102 (+ 1856us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:52.130295 (+ 3193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.130299 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.130301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.130661 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.131055 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.131058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.131060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.131189 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.131765 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:52.131768 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:52.131893 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 12:14:52.132014 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 12:14:52.132267 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 12:14:52.132718 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.120429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.4.48.252:35813)
0811 12:14:52.120591 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.120597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.120617 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:14:52.120847 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.120852 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.120920 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.121014 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.121738 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.122648 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.123861 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.124363 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.124455 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.127292 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.127298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.127302 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.127336 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.130104 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.130784 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.130790 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.130793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.130939 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.131290 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.131294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.131297 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.131618 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.131810 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.132109 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.132427 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":105,"server-negotiator.queue_time_us":51}
I20260811 12:14:52.136613 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.122506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40596 (local address 127.4.48.253:43113)
0811 12:14:52.122644 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.122647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.122661 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:14:52.122698 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.122703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.122795 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.122891 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.124057 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.124944 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.126096 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.126503 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.126618 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.131221 (+ 4603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.131226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.131229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.131265 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.134173 (+ 2908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.134764 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.134769 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.134771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.134902 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.135290 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.135295 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.135299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.135550 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.135833 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.136105 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.136323 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 12:14:52.137527 10179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: Bootstrap starting.
I20260811 12:14:52.140674 10179 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:52.140993 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:52.141705 10179 log.cc:824] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:52.144562 10179 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: No bootstrap required, opened a new log
I20260811 12:14:52.144091 9872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "460f370c1f844bf68aaefdd1706c866f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56136
I20260811 12:14:52.151469 10179 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.151790 10179 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f697fcc60bc439a9aa2925521fee89e, State: Initialized, Role: FOLLOWER
I20260811 12:14:52.152213 10179 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.153050 10189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.153352 10189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.154289 10179 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:14:52.163348 10203 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:14:52.163471 10203 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:14:52.161240 10064 master_runner.cc:186] Error getting master registration for 127.4.48.254:40159: OK, instance_id { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" instance_seqno: 1786450491067770 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:14:52.171860 10064 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:14:52.199007 9921 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 4
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:14:52.215728 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.199910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.4.48.252:35813)
0811 12:14:52.200112 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.200117 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.200144 (+ 27us) server_negotiation.cc:407] Connection header received
0811 12:14:52.200394 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.200401 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.200498 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.200606 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.201664 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.202887 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.204836 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.205650 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.205791 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.209608 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.209614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.209617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.209656 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.212918 (+ 3262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.213578 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.213582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.213585 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.213720 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.214122 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.214126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.214128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.214493 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.214680 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.215093 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.215327 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:14:52.215728 9924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.199853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56382)
0811 12:14:52.200073 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.200095 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:52.200229 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:52.200657 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:52.200662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:52.201274 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:52.201506 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.201520 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.203079 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.203085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.204661 (+ 1576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.204679 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.205821 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.205828 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.206032 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.206682 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:52.206705 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:52.209410 (+ 2705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:52.213048 (+ 3638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.213055 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.213060 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.213443 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.213838 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.213844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.213848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.214000 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.214630 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:52.214636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:52.214819 (+ 183us) client_negotiation.cc:769] Sending connection context
0811 12:14:52.214943 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 12:14:52.215241 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 12:14:52.221634 9921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: Bootstrap starting.
I20260811 12:14:52.225623 9921 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:52.226922 9921 log.cc:824] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:52.230129 9921 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: No bootstrap required, opened a new log
I20260811 12:14:52.237658 9921 raft_consensus.cc:348] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.238039 9921 raft_consensus.cc:729] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460f370c1f844bf68aaefdd1706c866f, State: Initialized, Role: FOLLOWER
I20260811 12:14:52.238530 9921 consensus_queue.cc:260] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.239483 10208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.239866 10208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.240739 9921 sys_catalog.cc:564] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:52.240890 9872 raft_consensus.cc:482] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:14:52.241072 9872 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:52.243927 9872 raft_consensus.cc:504] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.246423 10134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "460f370c1f844bf68aaefdd1706c866f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6f697fcc60bc439a9aa2925521fee89e"
I20260811 12:14:52.246893 10134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:52.250233 10134 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 460f370c1f844bf68aaefdd1706c866f in term 1.
I20260811 12:14:52.250779 9836 leader_election.cc:302] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460f370c1f844bf68aaefdd1706c866f, 6f697fcc60bc439a9aa2925521fee89e; no voters:
I20260811 12:14:52.251214 9872 leader_election.cc:288] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [CANDIDATE]: Term 1 election: Requested vote from peers 19781debdc9c4d9db76f86cb40ff2498 (127.4.48.253:43113), 6f697fcc60bc439a9aa2925521fee89e (127.4.48.252:35813)
I20260811 12:14:52.251247 10208 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:52.262804 10208 raft_consensus.cc:686] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [term 1 LEADER]: Becoming Leader. State: Replica: 460f370c1f844bf68aaefdd1706c866f, State: Running, Role: LEADER
I20260811 12:14:52.263899 10208 consensus_queue.cc:237] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.269459 10227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460f370c1f844bf68aaefdd1706c866f. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.269805 10227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:52.270960 10242 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:52.281440 10244 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:14:52.285555 10242 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:14:52.294144 10052 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } attempt: 5
I20260811 12:14:52.285387 9811 master_runner.cc:186] Error getting master registration for 127.4.48.253:43113: OK, instance_id { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" instance_seqno: 1786450491568810 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:14:52.303232 9811 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 12:14:52.306253 10097 tablet.cc:2369] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 12:14:52.309127 9839 tablet.cc:2369] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 12:14:52 dist-test-slave-nj21 krb5kdc[9787](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450491, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:14:52.314262 9926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.279244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.4.48.254:40159)
0811 12:14:52.279428 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.279434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.288185 (+ 8751us) server_negotiation.cc:407] Connection header received
0811 12:14:52.288455 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.288461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.288540 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.288618 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.291816 (+ 3198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.292992 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.296322 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.297079 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.297174 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.301914 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.301921 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.301926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.301985 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.305607 (+ 3622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.310623 (+ 5016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.310632 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.310636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.310796 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.311469 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.311474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.311477 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.311866 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.312107 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.313274 (+ 1167us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.313666 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:14:52.320518 10056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.294646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:35813 (local address 127.0.0.1:56406)
0811 12:14:52.294945 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:52.294962 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:52.295050 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:52.304363 (+ 9313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:52.304369 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:52.304892 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:52.305103 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.305116 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.306743 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.306750 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.308188 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:52.308204 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.309285 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.309291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:52.309530 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.310153 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:52.310177 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:52.313289 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:52.317687 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.317692 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.317696 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.318226 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.318720 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:52.318726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:52.318730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:52.318909 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:52.319542 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:52.319548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:52.319743 (+ 195us) client_negotiation.cc:769] Sending connection context
0811 12:14:52.319875 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 12:14:52.320140 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194}
I20260811 12:14:52.321877 10254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.295243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.4.48.252:35813)
0811 12:14:52.304094 (+ 8851us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.304098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.304117 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:14:52.304152 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.304155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.304201 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.304304 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.305300 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.306569 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.308402 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.309105 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.313460 (+ 4355us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.313692 (+ 232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.313699 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.313704 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.313743 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.317534 (+ 3791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.318386 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.318391 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.318395 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.318575 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.319059 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.319064 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.319068 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.319394 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.321096 (+ 1702us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.321220 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.321429 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8737,"thread_start_us":8533,"threads_started":1}
I20260811 12:14:52.327893 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: Bootstrap starting.
I20260811 12:14:52.330708 10134 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [term 1 FOLLOWER]: Refusing update from remote peer 460f370c1f844bf68aaefdd1706c866f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:52.331851 10208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:52.333271 10052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:52.333669 10058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.288879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40610 (local address 127.4.48.253:43113)
0811 12:14:52.289041 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.289047 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.292780 (+ 3733us) server_negotiation.cc:407] Connection header received
0811 12:14:52.293971 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.293978 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.294057 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.294129 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.304666 (+ 10537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.305753 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.318661 (+ 12908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.319322 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.319430 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.326599 (+ 7169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.326605 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.326609 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.326652 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.330297 (+ 3645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.331083 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.331088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.331091 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.331268 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.331819 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.331824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.331828 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.332194 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.332445 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.333000 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.333229 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 12:14:52.334837 10052 log.cc:824] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:52.343194 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:52.293209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56392 (local address 127.4.48.252:35813)
0811 12:14:52.293391 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.293401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.302277 (+ 8876us) server_negotiation.cc:407] Connection header received
0811 12:14:52.302669 (+ 392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.302674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.302781 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.302851 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.307771 (+ 4920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.308836 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.322732 (+ 13896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:52.323392 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:52.323487 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:52.335586 (+ 12099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:52.335593 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:52.335601 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:52.335641 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:52.339227 (+ 3586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.340150 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.340155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.340159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.340305 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:52.340766 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:52.340771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:52.340775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:52.341134 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:52.341439 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:52.342583 (+ 1144us) server_negotiation.cc:299] Negotiation successful
0811 12:14:52.342822 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:14:52.348827 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: No bootstrap required, opened a new log
I20260811 12:14:52.365038 10052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.366179 10052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19781debdc9c4d9db76f86cb40ff2498, State: Initialized, Role: FOLLOWER
I20260811 12:14:52.368062 10189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460f370c1f844bf68aaefdd1706c866f. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.368353 10189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.368690 10052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } }
I20260811 12:14:52.371127 10255 mvcc.cc:205] Tried to move back new op lower bound from 7317301216570761216 to 7317301216322338816. Current Snapshot: MvccSnapshot[applied={T|T < 7317301216570761216}]
I20260811 12:14:52.379580 10261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.381249 10189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.381529 10189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.381548 10261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.384229 10227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.384510 10227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:52.380144 9998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:52.380218 9999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "460f370c1f844bf68aaefdd1706c866f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "19781debdc9c4d9db76f86cb40ff2498"
I20260811 12:14:52.386106 10242 catalog_manager.cc:1347] Generated new cluster ID: 730a85e3bf114cc59f2b88fcac011fea
I20260811 12:14:52.386199 10242 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:52.388298 9998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [term 1 FOLLOWER]: Refusing update from remote peer 460f370c1f844bf68aaefdd1706c866f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:52.379766 10052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:52.389184 10208 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [LEADER]: Connected to new peer: Peer: permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:52.399075 10242 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:52.400945 10242 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:52.424089 10242 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f: Generated new TSK 0
I20260811 12:14:52.425068 10242 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:52.434557 9999 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 460f370c1f844bf68aaefdd1706c866f for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 12:14:52.440224 10261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 460f370c1f844bf68aaefdd1706c866f. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.440527 10261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.441085 10275 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.441572 10275 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:52.442662 10227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "460f370c1f844bf68aaefdd1706c866f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "460f370c1f844bf68aaefdd1706c866f" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 40159 } } peers { permanent_uuid: "19781debdc9c4d9db76f86cb40ff2498" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 43113 } } peers { permanent_uuid: "6f697fcc60bc439a9aa2925521fee89e" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 35813 } } }
I20260811 12:14:52.442920 10227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 460f370c1f844bf68aaefdd1706c866f [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:52.474303 10303 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260811 12:14:52.526641 9926 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.523741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.4.48.254:40159)
0811 12:14:52.523984 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.523991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.524579 (+ 588us) server_negotiation.cc:407] Connection header received
0811 12:14:52.524909 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.524915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.525025 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.525106 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.526362 (+ 1256us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56150: BlockingRecv error: recv got EOF from 127.0.0.1:56150 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 12:14:52.529232 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.524665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40618 (local address 127.4.48.253:43113)
0811 12:14:52.524850 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.524855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.527496 (+ 2641us) server_negotiation.cc:407] Connection header received
0811 12:14:52.527798 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.527804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.527903 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.527980 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.528996 (+ 1016us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40618: BlockingRecv error: recv got EOF from 127.0.0.1:40618 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 12:14:52.532450 10185 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.530505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56418 (local address 127.4.48.252:35813)
0811 12:14:52.530679 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.530685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.530865 (+ 180us) server_negotiation.cc:407] Connection header received
0811 12:14:52.531031 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.531036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.531117 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.531230 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.532155 (+ 925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56418: BlockingRecv error: recv got EOF from 127.0.0.1:56418 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 12:14:52.540162 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.537003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40622 (local address 127.4.48.253:43113)
0811 12:14:52.537154 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.537160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.538740 (+ 1580us) server_negotiation.cc:407] Connection header received
0811 12:14:52.538939 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.538944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.539069 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.539169 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.539958 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40622: BlockingRecv error: recv got EOF from 127.0.0.1:40622 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 12:14:52.544690 10049 acceptor_pool.cc:285] unable to collect diagnostics on RPC socket 127.4.48.253:43113: Invalid argument: socket address is ambiguous
W20260811 12:14:52.546653 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.544981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.4.48.253:43113)
0811 12:14:52.545186 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.545192 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.545397 (+ 205us) server_negotiation.cc:407] Connection header received
0811 12:14:52.545572 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.545578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.545652 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.545754 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.546472 (+ 718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40632: BlockingRecv error: recv got EOF from 127.0.0.1:40632 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 12:14:52.557449 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.552226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.4.48.253:43113)
0811 12:14:52.552384 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.552389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.556169 (+ 3780us) server_negotiation.cc:407] Connection header received
0811 12:14:52.556345 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.556350 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.556437 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.556532 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.557249 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40644: BlockingRecv error: recv got EOF from 127.0.0.1:40644 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 12:14:52.566262 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.564358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.4.48.253:43113)
0811 12:14:52.564527 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.564531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.565018 (+ 487us) server_negotiation.cc:407] Connection header received
0811 12:14:52.565183 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.565189 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.565264 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.565360 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.566073 (+ 713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40654: BlockingRecv error: recv got EOF from 127.0.0.1:40654 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:14:52.593276 10042 messenger.cc:188] Invalid argument: socket address is ambiguous: no data on pending connections for acceptor pool at 127.4.48.253:43113
W20260811 12:14:52.606351 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.604237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40666 (local address 127.4.48.253:43113)
0811 12:14:52.604433 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.604439 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.604955 (+ 516us) server_negotiation.cc:407] Connection header received
0811 12:14:52.605145 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.605151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.605239 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.605344 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.606144 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40666: BlockingRecv error: recv got EOF from 127.0.0.1:40666 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 12:14:52.627629 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.625482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40670 (local address 127.4.48.253:43113)
0811 12:14:52.625639 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.625644 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.626197 (+ 553us) server_negotiation.cc:407] Connection header received
0811 12:14:52.626425 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.626431 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.626528 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.626621 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.627413 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40670: BlockingRecv error: recv got EOF from 127.0.0.1:40670 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 12:14:52.664693 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.662622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40676 (local address 127.4.48.253:43113)
0811 12:14:52.662839 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.662845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.663222 (+ 377us) server_negotiation.cc:407] Connection header received
0811 12:14:52.663422 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.663426 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.663498 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.663579 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.664438 (+ 859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40676: BlockingRecv error: recv got EOF from 127.0.0.1:40676 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 12:14:52.734793 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.732682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40692 (local address 127.4.48.253:43113)
0811 12:14:52.732929 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.732935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.733372 (+ 437us) server_negotiation.cc:407] Connection header received
0811 12:14:52.733610 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.733614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.733685 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.733797 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.734575 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40692: BlockingRecv error: recv got EOF from 127.0.0.1:40692 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 12:14:52.867885 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:52.865962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.4.48.253:43113)
0811 12:14:52.866163 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:52.866170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:52.866479 (+ 309us) server_negotiation.cc:407] Connection header received
0811 12:14:52.866681 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:52.866686 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:52.866759 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:52.866841 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:52.867635 (+ 794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40704: BlockingRecv error: recv got EOF from 127.0.0.1:40704 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:14:53.129390 10058 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:53.127356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40706 (local address 127.4.48.253:43113)
0811 12:14:53.127550 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.127556 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.127881 (+ 325us) server_negotiation.cc:407] Connection header received
0811 12:14:53.128115 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.128120 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.128193 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.128276 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:53.129170 (+ 894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40706: BlockingRecv error: recv got EOF from 127.0.0.1:40706 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 12:14:53.166115 10203 catalog_manager.cc:1259] Loaded cluster ID: 730a85e3bf114cc59f2b88fcac011fea
I20260811 12:14:53.166251 10203 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: loading cluster ID for follower catalog manager: success
I20260811 12:14:53.170203 10203 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: acquiring CA information for follower catalog manager: success
I20260811 12:14:53.172746 10203 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f697fcc60bc439a9aa2925521fee89e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:14:53.478679 10303 catalog_manager.cc:1259] Loaded cluster ID: 730a85e3bf114cc59f2b88fcac011fea
I20260811 12:14:53.478796 10303 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: loading cluster ID for follower catalog manager: success
I20260811 12:14:53.481381 10303 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: acquiring CA information for follower catalog manager: success
I20260811 12:14:53.483091 10303 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19781debdc9c4d9db76f86cb40ff2498: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:14:53.651611 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:53.645256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40720 (local address 127.4.48.253:43113)
0811 12:14:53.645876 (+ 620us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.645882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.646164 (+ 282us) server_negotiation.cc:407] Connection header received
0811 12:14:53.646385 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.646391 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.646560 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.646675 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:53.647153 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.648006 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.649326 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.649790 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.649899 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:53.650426 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:53.650987 (+ 561us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:53.651131 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:53.651231 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 12:14:53.651312 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":316,"threads_started":1}
I20260811 12:14:53.668977 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:53.657680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56424 (local address 127.4.48.252:35813)
0811 12:14:53.663972 (+ 6292us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.663976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.663991 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:14:53.664079 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.664082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.664228 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.664366 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:53.664789 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.665505 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.666784 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.667175 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.667279 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:53.667762 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:53.668362 (+ 600us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:53.668500 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:53.668595 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:14:53.668678 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6193,"thread_start_us":6119,"threads_started":1}
I20260811 12:14:53.691354 10430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:53.675046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56162 (local address 127.4.48.254:40159)
0811 12:14:53.680728 (+ 5682us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.680736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.682700 (+ 1964us) server_negotiation.cc:407] Connection header received
0811 12:14:53.683172 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.683179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.683665 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.684502 (+ 837us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:53.684820 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.685913 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.688405 (+ 2492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.689080 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.689179 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:53.690056 (+ 877us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:53.690540 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:53.690622 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:53.690919 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 12:14:53.691018 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5531,"thread_start_us":6746,"threads_started":1}
I20260811 12:14:53.693786 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:53.684556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.4.48.253:43113)
0811 12:14:53.684747 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.684753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.686605 (+ 1852us) server_negotiation.cc:407] Connection header received
0811 12:14:53.686881 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.686887 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.686976 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.687036 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:53.687865 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.688946 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.691202 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.691854 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.691950 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:53.692981 (+ 1031us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:53.693109 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:53.693171 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:53.693411 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 12:14:53.693489 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 12:14:53.698189 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:53.686807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56440 (local address 127.4.48.252:35813)
0811 12:14:53.686996 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:53.687001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:53.692032 (+ 5031us) server_negotiation.cc:407] Connection header received
0811 12:14:53.692268 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:53.692274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:53.692361 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:53.692417 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:53.693083 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.694201 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.696081 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:53.696682 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:53.696804 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:53.697438 (+ 634us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:53.697562 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:53.697670 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:53.697835 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:14:53.697914 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 12:14:53.700703 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9811
I20260811 12:14:53.701946 9911 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:53.842012 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9811
W20260811 12:14:53.869580 9954 connection.cc:535] server connection from 127.0.0.1:40586 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 12:14:53.870486 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 9931
I20260811 12:14:53.871592 10038 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:53.974915 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 9931
I20260811 12:14:53.999832 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10064
I20260811 12:14:54.000959 10171 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:54.104662 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10064
2026-08-11T12:14:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4997 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:14:54 dist-test-slave-nj21 krb5kdc[10441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:14:54 dist-test-slave-nj21 krb5kdc[10441](info): set up 2 sockets
Aug 11 12:14:54 dist-test-slave-nj21 krb5kdc[10441](info): commencing operation
krb5kdc: starting...
W20260811 12:14:55.364270 4291 mini_kdc.cc:121] Time spent starting KDC: real 1.214s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/test-user.keytab.
Aug 11 12:14:55 dist-test-slave-nj21 krb5kdc[10441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.4.48.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
Entry for principal kudu/127.4.48.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab.
I20260811 12:14:55.634410 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38589
--webserver_interface=127.4.48.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:14:55.968885 10462 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:55.969200 10462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:55.969302 10462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:55.969405 10462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:55.976639 10462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:55.976773 10462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:55.976869 10462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:55.976936 10462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:55.977020 10462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:55.987224 10462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.254.keytab
--principal=kudu/127.4.48.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.254:38589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.4.48.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:55.988504 10462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:55.989802 10462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:56.001236 10469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.001225 10471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.001317 10468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:56.001977 10462 server_base.cc:1034] running on GCE node
I20260811 12:14:56.002650 10462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:56.003921 10462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:56.005139 10462 hybrid_clock.cc:648] HybridClock initialized: now 1786450496005070 us; error 37 us; skew 500 ppm
I20260811 12:14:56.005440 10462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:56.008404 10462 init.cc:377] Logged in from keytab as kudu/127.4.48.254@KRBTEST.COM (short username kudu)
I20260811 12:14:56.009800 10462 webserver.cc:460] Webserver started at http://127.4.48.254:33741/ using document root <none> and password file <none>
I20260811 12:14:56.010434 10462 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:56.010535 10462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:56.010850 10462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:56.013768 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data/instance:
uuid: "e35463371f5646a0a87fb6cddfbf7fad"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "dfac7680ede2e8979e0c4c22f1ef7f3f"
server_key_iv: "087f3d59385a9193765c539af2ad5b42"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.014528 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance:
uuid: "e35463371f5646a0a87fb6cddfbf7fad"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "dfac7680ede2e8979e0c4c22f1ef7f3f"
server_key_iv: "087f3d59385a9193765c539af2ad5b42"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.019485 10462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260811 12:14:56.022718 10482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:56.023976 10462 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 12:14:56.024139 10462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal
uuid: "e35463371f5646a0a87fb6cddfbf7fad"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "dfac7680ede2e8979e0c4c22f1ef7f3f"
server_key_iv: "087f3d59385a9193765c539af2ad5b42"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.024330 10462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:56.035650 10462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:56.039340 10462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:56.039589 10462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:56.056962 10462 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.254:38589
I20260811 12:14:56.056979 10577 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.254:38589 every 8 connection(s)
I20260811 12:14:56.058318 10462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/data/info.pb
I20260811 12:14:56.058764 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10462
I20260811 12:14:56.058969 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-0/wal/instance
I20260811 12:14:56.059538 4291 external_mini_cluster.cc:1428] Setting key f5865caac7c8c2bdb4266608dbc55515
I20260811 12:14:56.063023 10578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:56.071377 10578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.4.48.253@KRBTEST.COM; defaulting to no policy
I20260811 12:14:56.093905 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.074860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55438 (local address 127.4.48.254:38589)
0811 12:14:56.075653 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:56.075667 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:56.076127 (+ 460us) server_negotiation.cc:407] Connection header received
0811 12:14:56.078016 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:56.078021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:56.078228 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:56.078400 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:56.079811 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.081582 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.082927 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.083785 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.083961 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.087232 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:56.087240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:56.087251 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:56.087279 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:56.089931 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.090739 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.090745 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.090747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.090881 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.091323 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.091326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.091328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.091810 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:56.091970 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:56.092607 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 12:14:56.092853 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":572,"thread_start_us":351,"threads_started":1}
I20260811 12:14:56.093915 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.074861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38589 (local address 127.0.0.1:55438)
0811 12:14:56.075979 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.076014 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:56.077679 (+ 1665us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:56.078452 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:56.078460 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:56.079100 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:56.079526 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.079552 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.081743 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.081747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.082790 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.082801 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.083992 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.083997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.084117 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.084689 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:56.084715 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:56.086999 (+ 2284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:56.090061 (+ 3062us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.090073 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.090085 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.090621 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.091016 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.091021 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.091025 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.091222 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.091969 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:56.091977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:56.092238 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 12:14:56.092487 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 12:14:56.092980 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":161,"thread_start_us":717,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.253@KRBTEST.COM" created.
I20260811 12:14:56.096619 10578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:56.097643 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.097280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33324)
0811 12:14:56.097442 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.097547 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 12:14:56.097991 10484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
I20260811 12:14:56.099411 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.099100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33328)
0811 12:14:56.099227 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.099291 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:14:56.099594 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:38551: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
Entry for principal kudu/127.4.48.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab.
I20260811 12:14:56.128237 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:38551
--webserver_interface=127.4.48.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:14:56.148912 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } attempt: 1
I20260811 12:14:56.150041 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.149603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33338)
0811 12:14:56.149802 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.149894 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:14:56.151137 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.150797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33350)
0811 12:14:56.150942 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.151009 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:14:56.151340 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:38551: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
I20260811 12:14:56.219561 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } attempt: 2
I20260811 12:14:56.220752 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.220301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33366)
0811 12:14:56.220526 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.220614 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:14:56.221701 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.221382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33376)
0811 12:14:56.221492 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.221570 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 12:14:56.221890 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:38551: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
I20260811 12:14:56.313145 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } attempt: 3
I20260811 12:14:56.314457 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.313969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33392)
0811 12:14:56.314188 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.314295 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 12:14:56.315531 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.315142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33408)
0811 12:14:56.315295 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.315376 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:14:56.315766 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:38551: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
I20260811 12:14:56.458997 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } attempt: 4
W20260811 12:14:56.459306 10586 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:56.459604 10586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:56.459710 10586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:56.459834 10586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 12:14:56.460186 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.459554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33416)
0811 12:14:56.459919 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.460027 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188}
I20260811 12:14:56.461189 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.460803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33432)
0811 12:14:56.460951 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.461048 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 12:14:56.461388 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.253:38551: Network error: Client connection negotiation failed: client connection to 127.4.48.253:38551: connect: Connection refused (error 111)
W20260811 12:14:56.467095 10586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:56.467175 10586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:56.467265 10586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:56.467327 10586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:56.467414 10586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:56.476732 10586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.253.keytab
--principal=kudu/127.4.48.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.253:38551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.4.48.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:56.477917 10586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:56.479146 10586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:56.490257 10594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.490342 10597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.490316 10595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:56.491086 10586 server_base.cc:1034] running on GCE node
I20260811 12:14:56.491816 10586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:56.493139 10586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:56.494347 10586 hybrid_clock.cc:648] HybridClock initialized: now 1786450496494277 us; error 39 us; skew 500 ppm
I20260811 12:14:56.494647 10586 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:56.497587 10586 init.cc:377] Logged in from keytab as kudu/127.4.48.253@KRBTEST.COM (short username kudu)
I20260811 12:14:56.498914 10586 webserver.cc:460] Webserver started at http://127.4.48.253:35183/ using document root <none> and password file <none>
I20260811 12:14:56.499518 10586 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:56.499601 10586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:56.499923 10586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:56.502683 10586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data/instance:
uuid: "1e912bf63431420089dbd07a81555a82"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "aff2b71045955ae30e1b756c0d15e6f8"
server_key_iv: "877b23d8895d36104f6a4606f36436bd"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.503401 10586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal/instance:
uuid: "1e912bf63431420089dbd07a81555a82"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "aff2b71045955ae30e1b756c0d15e6f8"
server_key_iv: "877b23d8895d36104f6a4606f36436bd"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.508144 10586 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 12:14:56.511274 10610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:56.512533 10586 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 12:14:56.512689 10586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal
uuid: "1e912bf63431420089dbd07a81555a82"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "aff2b71045955ae30e1b756c0d15e6f8"
server_key_iv: "877b23d8895d36104f6a4606f36436bd"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.512867 10586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:56.533109 10586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:56.536885 10586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:56.537138 10586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:56.554895 10586 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.253:38551
I20260811 12:14:56.554926 10702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.253:38551 every 8 connection(s)
I20260811 12:14:56.556157 10586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/data/info.pb
I20260811 12:14:56.557327 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10586
I20260811 12:14:56.557514 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-1/wal/instance
I20260811 12:14:56.558087 4291 external_mini_cluster.cc:1428] Setting key 85d89d3a6fbf70c924315f46273fccd2
I20260811 12:14:56.560565 10703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:56.568585 10703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:56.589222 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.571397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55446 (local address 127.4.48.254:38589)
0811 12:14:56.571600 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:56.571606 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:56.572786 (+ 1180us) server_negotiation.cc:407] Connection header received
0811 12:14:56.575040 (+ 2254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:56.575046 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:56.575128 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:56.575227 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:56.576664 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.577653 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.579076 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.579731 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.579858 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.583127 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:56.583133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:56.583137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:56.583176 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:56.586244 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.586897 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.586902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.586905 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.587057 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.587462 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.587468 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.587471 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.587900 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:56.588057 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:56.588654 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 12:14:56.588898 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 12:14:56.589848 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.571872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38589 (local address 127.0.0.1:55446)
0811 12:14:56.572632 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.572677 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:56.574755 (+ 2078us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:56.575415 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:56.575423 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:56.575976 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:56.576491 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.576508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.577780 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.577784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.578922 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.578933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.579934 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.579939 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.580130 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.580757 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:56.580778 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:56.582906 (+ 2128us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:56.586369 (+ 3463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.586376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.586386 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.586779 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.587178 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.587181 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.587184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.587357 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.588017 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:56.588023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:56.588294 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 12:14:56.588514 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 12:14:56.588838 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for kudu/127.4.48.252@KRBTEST.COM; defaulting to no policy
I20260811 12:14:56.591456 10703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.4.48.252@KRBTEST.COM" created.
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:56.604568 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.592175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33434)
0811 12:14:56.592374 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.592390 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:56.592471 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:56.593314 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:56.593318 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:56.593689 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:56.593812 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.593819 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.595271 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.595275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.596133 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.596145 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.597025 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.597031 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.597184 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.597646 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:56.597662 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:56.599613 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:56.602357 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.602362 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.602366 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.602676 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.603002 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.603007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.603011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.603136 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.603707 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:56.603711 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:56.603848 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 12:14:56.603961 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 12:14:56.604206 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:14:56.604677 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.592367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.4.48.253:38551)
0811 12:14:56.592961 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:56.592966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:56.592988 (+ 22us) server_negotiation.cc:407] Connection header received
0811 12:14:56.593026 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:56.593032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:56.593175 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:56.593300 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:56.593968 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.595145 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.596261 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.596898 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.597004 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.599808 (+ 2804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:56.599818 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:56.599831 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:56.599858 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:56.602241 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.602788 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.602795 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.602797 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.602900 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.603229 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.603234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.603237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.603551 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:56.603761 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:56.604117 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 12:14:56.604321 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":321,"threads_started":1}
I20260811 12:14:56.606658 10703 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:56.607633 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.607159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36474)
0811 12:14:56.607333 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.607469 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:14:56.607915 10611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.609368 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.609008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36482)
0811 12:14:56.609158 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.609251 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 12:14:56.609603 10703 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
Entry for principal kudu/127.4.48.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab.
I20260811 12:14:56.626636 4291 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
/tmp/dist-test-taskXBhqNr/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:33601
--webserver_interface=127.4.48.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.4.48.212:40463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:14:56.658875 10703 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 1
I20260811 12:14:56.659986 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.659518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36498)
0811 12:14:56.659734 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.659840 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:14:56.660924 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.660609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36510)
0811 12:14:56.660731 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.660817 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 12:14:56.661116 10703 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.729296 10703 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 2
I20260811 12:14:56.730414 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.729979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36526)
0811 12:14:56.730153 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.730248 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 12:14:56.731344 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.731046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36532)
0811 12:14:56.731157 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.731221 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 12:14:56.731539 10703 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.760617 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } attempt: 5
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:56.773233 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.761326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33450)
0811 12:14:56.761531 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.761551 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:56.761688 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:56.762047 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:56.762051 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:56.762453 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:56.762632 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.762641 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.763625 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.763629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.764515 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:56.764527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.765265 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.765271 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:56.765486 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.766082 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:56.766100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:56.768238 (+ 2138us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:56.770967 (+ 2729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.770972 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.770976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.771335 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.771695 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:56.771700 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:56.771704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:56.771843 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:56.772346 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:56.772351 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:56.772498 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 12:14:56.772610 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 12:14:56.772869 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 12:14:56.773257 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.761371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.4.48.253:38551)
0811 12:14:56.761535 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:56.761541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:56.761627 (+ 86us) server_negotiation.cc:407] Connection header received
0811 12:14:56.761814 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:56.761819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:56.761905 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:56.761993 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:56.762789 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.763483 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.764671 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:56.765123 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:56.765233 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:56.768414 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:56.768419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:56.768422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:56.768455 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:56.770823 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.771444 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.771448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.771451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.771571 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:56.771943 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:56.771948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:56.771951 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:56.772221 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:56.772355 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:56.772745 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 12:14:56.772965 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 12:14:56.774222 10578 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:56.775211 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.774856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36544)
0811 12:14:56.775005 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.775075 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 12:14:56.776160 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.775867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36560)
0811 12:14:56.775986 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.776068 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 12:14:56.776353 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.822759 10703 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 3
I20260811 12:14:56.823963 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.823432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36568)
0811 12:14:56.823652 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.823788 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 12:14:56.825039 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.824598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36582)
0811 12:14:56.824825 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.824911 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 12:14:56.825263 10703 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.827541 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 1
I20260811 12:14:56.828577 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.828133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36596)
0811 12:14:56.828319 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.828402 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 12:14:56.829581 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.829244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36608)
0811 12:14:56.829392 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.829475 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:14:56.829824 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.898033 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 2
I20260811 12:14:56.899084 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.898637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36620)
0811 12:14:56.898839 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.898927 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:14:56.900080 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.899760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36632)
0811 12:14:56.899899 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.899967 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 12:14:56.900269 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
W20260811 12:14:56.952802 10712 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:14:56.953110 10712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:14:56.953207 10712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:14:56.953321 10712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:14:56.960304 10712 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:14:56.960402 10712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:14:56.960512 10712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:14:56.960590 10712 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:14:56.960675 10712 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:14:56.968472 10703 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 4
I20260811 12:14:56.969565 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.969103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36646)
0811 12:14:56.969315 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.969404 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 12:14:56.970269 10712 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.4.48.212:40463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.4.48.254:38589,127.4.48.253:38551,127.4.48.252:33601
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/krb5kdc/kudu_127.4.48.252.keytab
--principal=kudu/127.4.48.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.4.48.252:33601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.4.48.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 10:48:20 UTC on ccycloud.quasar-kfysal.root.comops.site
build id 4580138
ASAN enabled
I20260811 12:14:56.970513 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:56.970172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36662)
0811 12:14:56.970326 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:56.970393 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:14:56.970750 10703 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:56.971481 10712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:14:56.972750 10712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:14:56.984380 10721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.984349 10718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:14:56.984355 10719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:14:56.985312 10712 server_base.cc:1034] running on GCE node
I20260811 12:14:56.986048 10712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:14:56.987287 10712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:14:56.988493 10712 hybrid_clock.cc:648] HybridClock initialized: now 1786450496988421 us; error 39 us; skew 500 ppm
I20260811 12:14:56.988791 10712 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 12:14:56 dist-test-slave-nj21 krb5kdc[10441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:14:56.991611 10712 init.cc:377] Logged in from keytab as kudu/127.4.48.252@KRBTEST.COM (short username kudu)
I20260811 12:14:56.992940 10712 webserver.cc:460] Webserver started at http://127.4.48.252:36821/ using document root <none> and password file <none>
I20260811 12:14:56.993554 10712 fs_manager.cc:359] Metadata directory not provided
I20260811 12:14:56.993659 10712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:14:56.993952 10712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:14:56.996716 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data/instance:
uuid: "19691e289d9248dc89a7cf35f075b019"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "395b0fd0a77af8c158c9c3c540458825"
server_key_iv: "965efa510883e48a1ad8df570fd06350"
server_key_version: "encryptionkey@0"
I20260811 12:14:56.997431 10712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal/instance:
uuid: "19691e289d9248dc89a7cf35f075b019"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "395b0fd0a77af8c158c9c3c540458825"
server_key_iv: "965efa510883e48a1ad8df570fd06350"
server_key_version: "encryptionkey@0"
I20260811 12:14:57.002115 10712 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 12:14:57.005231 10738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:14:57.006388 10712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 12:14:57.006459 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 3
I20260811 12:14:57.006544 10712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data,/tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal
uuid: "19691e289d9248dc89a7cf35f075b019"
format_stamp: "Formatted at 2026-08-11 12:14:56 on dist-test-slave-nj21"
server_key: "395b0fd0a77af8c158c9c3c540458825"
server_key_iv: "965efa510883e48a1ad8df570fd06350"
server_key_version: "encryptionkey@0"
I20260811 12:14:57.006750 10712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:14:57.007416 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.006962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36678)
0811 12:14:57.007165 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.007254 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 12:14:57.008332 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.007993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36684)
0811 12:14:57.008156 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.008221 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:14:57.008508 10578 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.4.48.252:33601: Network error: Client connection negotiation failed: client connection to 127.4.48.252:33601: connect: Connection refused (error 111)
I20260811 12:14:57.018676 10712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:14:57.022310 10712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:14:57.022539 10712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:14:57.039343 10712 rpc_server.cc:307] RPC server started. Bound to: 127.4.48.252:33601
I20260811 12:14:57.039362 10818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.4.48.252:33601 every 8 connection(s)
I20260811 12:14:57.040570 10712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/data/info.pb
I20260811 12:14:57.044020 10822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:14:57.050686 4291 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu as pid 10712
I20260811 12:14:57.050889 4291 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskXBhqNr/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786450132843697-4291-0/minicluster-data/master-2/wal/instance
I20260811 12:14:57.050207 10822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:57.051450 4291 external_mini_cluster.cc:1428] Setting key 137125fa8d50d2eb72e3e9ef6a6fa20f
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:14:57.072269 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.053241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55450 (local address 127.4.48.254:38589)
0811 12:14:57.053440 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.053457 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.054298 (+ 841us) server_negotiation.cc:407] Connection header received
0811 12:14:57.056087 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.056093 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.056180 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.056311 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.057998 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.059122 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.061382 (+ 2260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.062118 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.062265 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.065756 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.065763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.065767 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.065815 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.069037 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.069786 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.069792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.069795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.069986 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.070436 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.070439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.070441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.070740 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.070902 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.071691 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.071939 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 12:14:57.073228 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.053214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.254:38589 (local address 127.0.0.1:55450)
0811 12:14:57.054137 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.054175 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:57.055833 (+ 1658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:57.056835 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:57.056840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:57.057425 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:57.057817 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.057835 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.059346 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.059352 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.061204 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.061219 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.062351 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.062361 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.062526 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.063115 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:57.063139 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:57.065557 (+ 2418us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:57.069166 (+ 3609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.069175 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.069183 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.069651 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.070096 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.070100 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.070103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.070323 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.070866 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:57.070872 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:57.071245 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 12:14:57.071534 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 12:14:57.071973 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":653,"mutex_wait_us":49,"spinlock_wait_cycles":17280,"thread_start_us":412,"threads_started":1}
I20260811 12:14:57.073242 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.053230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36698 (local address 127.4.48.252:33601)
0811 12:14:57.054096 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.054111 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.054150 (+ 39us) server_negotiation.cc:407] Connection header received
0811 12:14:57.055832 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.055854 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.056123 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.056353 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.058094 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.059651 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.060999 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.061741 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.061922 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.066529 (+ 4607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.066694 (+ 165us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.066713 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.066759 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.070024 (+ 3265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.070726 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.070734 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.070737 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.070891 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.071300 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.071302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.071304 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.071722 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.072053 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.072590 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.072767 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":693,"thread_start_us":441,"threads_started":1}
I20260811 12:14:57.074793 10822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.254@KRBTEST.COM
I20260811 12:14:57.087142 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.075533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.4.48.253:38551)
0811 12:14:57.075719 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.075725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.075744 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:14:57.075836 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.075841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.075924 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.076020 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.076781 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.077443 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.078553 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.078967 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.079073 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.082002 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.082009 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.082013 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.082054 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.084642 (+ 2588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.085325 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.085330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.085333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.085511 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.085862 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.085866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.085870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.086149 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.086299 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.086721 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.086874 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:14:57.087227 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.075426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.253:38551 (local address 127.0.0.1:33462)
0811 12:14:57.075583 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.075600 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:57.075726 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:57.076070 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:57.076073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:57.076448 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:57.076634 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.076646 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.077581 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.077586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.078420 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.078431 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.079094 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.079100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.079263 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.079783 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:57.079798 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:57.081770 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:57.084753 (+ 2983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.084759 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.084762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.085228 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.085621 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.085626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.085630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.085761 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.086262 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:57.086266 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:57.086429 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 12:14:57.086622 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 12:14:57.086873 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 12:14:57.088109 10822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:14:57.090351 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.076364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55452 (local address 127.4.48.254:38589)
0811 12:14:57.076520 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.076523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.076537 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:14:57.076665 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.076668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.076717 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.076796 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.077901 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.078606 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.079922 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.080298 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.080405 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.084678 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.084682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.084685 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.084720 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.087869 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.088504 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.088509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.088511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.088653 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.089068 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.089071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.089073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.089402 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.089674 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.089892 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.090090 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.252@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450495, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.4.48.253@KRBTEST.COM
I20260811 12:14:57.100617 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.088662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36704)
0811 12:14:57.088809 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.088823 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:57.088887 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:57.089229 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:57.089233 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:57.089705 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:57.089879 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.089889 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.090989 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.090994 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.092116 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.092130 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.092751 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.092756 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.092947 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.093410 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:57.093425 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:57.095398 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:57.098148 (+ 2750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.098154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.098157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.098632 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.098959 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.098964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.098968 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.099127 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.099733 (+ 606us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:57.099738 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:57.099871 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 12:14:57.100000 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 12:14:57.100250 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 12:14:57.100706 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.088845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36704 (local address 127.4.48.252:33601)
0811 12:14:57.088983 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.088988 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.089004 (+ 16us) server_negotiation.cc:407] Connection header received
0811 12:14:57.089046 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.089051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.089111 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.089174 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.090030 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.090865 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.092239 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.092619 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.092730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.095566 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.095571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.095574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.095609 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.098015 (+ 2406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.098735 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.098738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.098741 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.098851 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.099231 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.099236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.099240 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.099584 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.099753 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.100101 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.100390 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":99,"server-negotiator.queue_time_us":50}
I20260811 12:14:57.106359 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.092031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.4.48.253:38551)
0811 12:14:57.092201 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.092206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.092221 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:14:57.092256 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.092259 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.092307 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.092383 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.093568 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.094228 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.095637 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.096290 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.096405 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.100552 (+ 4147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.100556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.100559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.100588 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.104040 (+ 3452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.104597 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.104602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.104606 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.104758 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.105094 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.105099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.105102 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.105426 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.105686 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.105840 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.106052 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:14:57.106511 10822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: Bootstrap starting.
I20260811 12:14:57.110116 10822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:57.110872 4291 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:14:57.111335 10822 log.cc:824] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:57.114274 10822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: No bootstrap required, opened a new log
I20260811 12:14:57.113579 10530 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e35463371f5646a0a87fb6cddfbf7fad"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55452
I20260811 12:14:57.124786 10822 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.125154 10822 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19691e289d9248dc89a7cf35f075b019, State: Initialized, Role: FOLLOWER
I20260811 12:14:57.125633 10822 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.126497 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.126792 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.127777 10822 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 12:14:57.136615 10853 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:14:57.136740 10853 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:14:57.134673 10712 master_runner.cc:186] Error getting master registration for 127.4.48.254:38589: OK, instance_id { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" instance_seqno: 1786450496051975 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:14:57.142392 10712 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:14:57.185702 10578 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 4
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.254@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:14:57.200201 10583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.186415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36718)
0811 12:14:57.186645 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.186666 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:57.186804 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:57.187133 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:57.187137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:57.187564 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:57.187783 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.187795 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.188800 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.188806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.189744 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.189756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.190544 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.190549 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.190738 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.191288 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:57.191305 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:57.193529 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:57.197461 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.197467 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.197471 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.197962 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.198344 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.198349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.198353 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.198490 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.199201 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:57.199205 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:57.199366 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 12:14:57.199509 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 12:14:57.199800 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 12:14:57.200526 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.186437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36718 (local address 127.4.48.252:33601)
0811 12:14:57.186613 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.186620 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.186760 (+ 140us) server_negotiation.cc:407] Connection header received
0811 12:14:57.186931 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.186935 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.186996 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.187083 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.187923 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.188645 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.189878 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.190386 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.190511 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.193743 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.193750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.193755 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.193797 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.197322 (+ 3525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.198073 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.198079 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.198082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.198229 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.198580 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.198586 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.198589 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.199072 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.199243 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.199622 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.199888 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 12:14:57.204893 10578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: Bootstrap starting.
I20260811 12:14:57.208806 10578 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:57.210067 10578 log.cc:824] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:57.213023 10578 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: No bootstrap required, opened a new log
I20260811 12:14:57.218279 10578 raft_consensus.cc:348] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.218609 10578 raft_consensus.cc:729] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35463371f5646a0a87fb6cddfbf7fad, State: Initialized, Role: FOLLOWER
I20260811 12:14:57.219038 10578 consensus_queue.cc:260] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.220005 10857 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.220427 10857 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.221248 10578 sys_catalog.cc:564] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:57.221346 10530 raft_consensus.cc:482] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:14:57.221539 10530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:57.224920 10530 raft_consensus.cc:504] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.227555 10783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e35463371f5646a0a87fb6cddfbf7fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "19691e289d9248dc89a7cf35f075b019"
I20260811 12:14:57.228145 10783 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:57.231942 10783 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35463371f5646a0a87fb6cddfbf7fad in term 1.
I20260811 12:14:57.232646 10484 leader_election.cc:302] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19691e289d9248dc89a7cf35f075b019, e35463371f5646a0a87fb6cddfbf7fad; no voters:
I20260811 12:14:57.233163 10530 leader_election.cc:288] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [CANDIDATE]: Term 1 election: Requested vote from peers 1e912bf63431420089dbd07a81555a82 (127.4.48.253:38551), 19691e289d9248dc89a7cf35f075b019 (127.4.48.252:33601)
I20260811 12:14:57.233837 10857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:14:57.240751 10857 raft_consensus.cc:686] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [term 1 LEADER]: Becoming Leader. State: Replica: e35463371f5646a0a87fb6cddfbf7fad, State: Running, Role: LEADER
I20260811 12:14:57.241382 10857 consensus_queue.cc:237] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.249409 10863 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35463371f5646a0a87fb6cddfbf7fad. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.249946 10863 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:57.256084 10872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:14:57.269943 10703 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } attempt: 5
I20260811 12:14:57.271031 10872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260811 12:14:57.275894 10749 tablet.cc:2369] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 12:14:57 dist-test-slave-nj21 krb5kdc[10441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786450496, etypes {rep=17 tkt=17 ses=17}, kudu/127.4.48.253@KRBTEST.COM for kudu/127.4.48.252@KRBTEST.COM
I20260811 12:14:57.288375 10706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.270498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.4.48.252:33601 (local address 127.0.0.1:36734)
0811 12:14:57.270866 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 12:14:57.270886 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 12:14:57.271006 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:14:57.272015 (+ 1009us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:14:57.272021 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:14:57.272580 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:14:57.272781 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.272794 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.274182 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.274188 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.275533 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:14:57.275548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.277579 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.277585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:14:57.277800 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.278482 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:14:57.278506 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:14:57.281245 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:14:57.285436 (+ 4191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.285445 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.285449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.285975 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.286445 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:14:57.286453 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:14:57.286457 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:14:57.286634 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:14:57.287333 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:14:57.287338 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:14:57.287503 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 12:14:57.287649 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 12:14:57.287947 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260811 12:14:57.289327 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.271338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36734 (local address 127.4.48.252:33601)
0811 12:14:57.271514 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.271520 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.271539 (+ 19us) server_negotiation.cc:407] Connection header received
0811 12:14:57.271602 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.271607 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.271728 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.271820 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.272960 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.273997 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.276654 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.277248 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.277380 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.281475 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.281482 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.281487 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.281525 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.285270 (+ 3745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.286123 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.286128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.286132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.286293 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.286766 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.286771 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.286775 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.287109 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.287243 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.288656 (+ 1413us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.288869 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 12:14:57.292207 10492 tablet.cc:2369] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 12:14:57.292829 10886 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 12:14:57.294015 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.261881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.4.48.253:38551)
0811 12:14:57.262078 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.262085 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.271713 (+ 9628us) server_negotiation.cc:407] Connection header received
0811 12:14:57.272058 (+ 345us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.272063 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.272127 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.272190 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.275654 (+ 3464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.276656 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.279572 (+ 2916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.280164 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.280259 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.285602 (+ 5343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.285608 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.285612 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.285652 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.289506 (+ 3854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.290992 (+ 1486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.290997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.291000 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.291158 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.291869 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.291874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.291877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.292242 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.292470 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.293475 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.293690 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 12:14:57.295444 10703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: Bootstrap starting.
I20260811 12:14:57.299445 10703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: Neither blocks nor log segments found. Creating new log.
I20260811 12:14:57.299760 10783 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [term 1 FOLLOWER]: Refusing update from remote peer e35463371f5646a0a87fb6cddfbf7fad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:57.300710 10863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:57.301101 10703 log.cc:824] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: Log is configured to *not* fsync() on all Append() calls
I20260811 12:14:57.299823 10462 master_runner.cc:186] Error getting master registration for 127.4.48.253:38551: OK, instance_id { permanent_uuid: "1e912bf63431420089dbd07a81555a82" instance_seqno: 1786450496550078 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:14:57.310207 10462 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:14:57.313256 10703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: No bootstrap required, opened a new log
I20260811 12:14:57.317605 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.262090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55464 (local address 127.4.48.254:38589)
0811 12:14:57.262275 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.262280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.295422 (+ 33142us) server_negotiation.cc:407] Connection header received
0811 12:14:57.295895 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.295901 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.295983 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.296052 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.298872 (+ 2820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.299971 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.303093 (+ 3122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.303828 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.303931 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.309144 (+ 5213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.309151 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.309156 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.309207 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.313122 (+ 3915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.314482 (+ 1360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.314488 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.314491 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.314665 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.315379 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.315384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.315388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.315813 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.316075 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.316992 (+ 917us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.317234 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 12:14:57.322228 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35463371f5646a0a87fb6cddfbf7fad. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.322553 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.338799 10703 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.339214 10703 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e912bf63431420089dbd07a81555a82, State: Initialized, Role: FOLLOWER
I20260811 12:14:57.339779 10703 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } }
I20260811 12:14:57.340749 10832 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.341009 10832 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.342088 10863 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.342360 10863 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:57.343966 10872 catalog_manager.cc:1347] Generated new cluster ID: acf8b387d9d849dd9598fb7203336c2b
I20260811 12:14:57.344064 10872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:14:57.346205 10895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.346621 10895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.347826 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.321020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36742 (local address 127.4.48.252:33601)
0811 12:14:57.321274 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.321312 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.321651 (+ 339us) server_negotiation.cc:407] Connection header received
0811 12:14:57.321980 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.322017 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.322813 (+ 796us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.323031 (+ 218us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.326887 (+ 3856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.329205 (+ 2318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.333067 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.334478 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.334709 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.340927 (+ 6218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:14:57.340933 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:14:57.340937 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:14:57.340974 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:14:57.344713 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.345538 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.345544 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.345547 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.345708 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:14:57.346100 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:14:57.346105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:14:57.346109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:14:57.346446 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:14:57.346694 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.347191 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.347449 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 12:14:57.348716 10703 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:14:57.349144 10661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:14:57.353080 10661 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [term 1 FOLLOWER]: Refusing update from remote peer e35463371f5646a0a87fb6cddfbf7fad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:14:57.353691 10662 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e35463371f5646a0a87fb6cddfbf7fad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1e912bf63431420089dbd07a81555a82"
I20260811 12:14:57.354014 10863 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:14:57.357424 10662 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e35463371f5646a0a87fb6cddfbf7fad in term 1.
I20260811 12:14:57.401036 10895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e35463371f5646a0a87fb6cddfbf7fad. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.401350 10895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.404588 10909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.404980 10909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:14:57.406491 10897 sys_catalog.cc:455] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e35463371f5646a0a87fb6cddfbf7fad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e35463371f5646a0a87fb6cddfbf7fad" member_type: VOTER last_known_addr { host: "127.4.48.254" port: 38589 } } peers { permanent_uuid: "1e912bf63431420089dbd07a81555a82" member_type: VOTER last_known_addr { host: "127.4.48.253" port: 38551 } } peers { permanent_uuid: "19691e289d9248dc89a7cf35f075b019" member_type: VOTER last_known_addr { host: "127.4.48.252" port: 33601 } } }
I20260811 12:14:57.406890 10897 sys_catalog.cc:458] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad [sys.catalog]: This master's current role is: LEADER
I20260811 12:14:57.407594 10872 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:14:57.409409 10872 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:14:57.429114 10872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e35463371f5646a0a87fb6cddfbf7fad: Generated new TSK 0
I20260811 12:14:57.430239 10872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 12:14:57.454453 10919 catalog_manager.cc:1259] Loaded cluster ID: acf8b387d9d849dd9598fb7203336c2b
I20260811 12:14:57.454624 10919 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: loading cluster ID for follower catalog manager: success
I20260811 12:14:57.458122 10919 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: acquiring CA information for follower catalog manager: success
I20260811 12:14:57.460443 10919 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1e912bf63431420089dbd07a81555a82: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 12:14:57.574088 10708 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.558757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33482 (local address 127.4.48.253:38551)
0811 12:14:57.558964 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.558972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.570268 (+ 11296us) server_negotiation.cc:407] Connection header received
0811 12:14:57.572509 (+ 2241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.572516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.572633 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.572706 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.573862 (+ 1156us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33482: BlockingRecv error: recv got EOF from 127.0.0.1:33482 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 12:14:57.577791 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.570754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.4.48.252:33601)
0811 12:14:57.570941 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.570948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.576087 (+ 5139us) server_negotiation.cc:407] Connection header received
0811 12:14:57.576243 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.576248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.576335 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.576450 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.577543 (+ 1093us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36756: BlockingRecv error: recv got EOF from 127.0.0.1:36756 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 12:14:57.579555 10582 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.567002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55474 (local address 127.4.48.254:38589)
0811 12:14:57.567198 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.567205 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.575333 (+ 8128us) server_negotiation.cc:407] Connection header received
0811 12:14:57.575498 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.575503 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.575608 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.575751 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.579323 (+ 3572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55474: BlockingRecv error: recv got EOF from 127.0.0.1:55474 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:14:57.623211 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.602796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33490 (local address 127.4.48.253:38551)
0811 12:14:57.602972 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.602978 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.615445 (+ 12467us) server_negotiation.cc:407] Connection header received
0811 12:14:57.616006 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.616012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.616210 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.616338 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.617096 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.618173 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.620248 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.620895 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.621001 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.621713 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.622387 (+ 674us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.622507 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.622766 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.622872 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 12:14:57.671236 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.658248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36766 (local address 127.4.48.252:33601)
0811 12:14:57.658448 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.658454 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.669056 (+ 10602us) server_negotiation.cc:407] Connection header received
0811 12:14:57.669617 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.669624 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.669797 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.669874 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.670969 (+ 1095us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36766: BlockingRecv error: recv got EOF from 127.0.0.1:36766 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:14:57.725848 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.707285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.4.48.253:38551)
0811 12:14:57.707516 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.707522 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.718581 (+ 11059us) server_negotiation.cc:407] Connection header received
0811 12:14:57.719147 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.719154 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.719281 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.719356 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.720127 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.721264 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.723349 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.724139 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.724238 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.724953 (+ 715us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.725118 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.725181 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.725427 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.725516 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 12:14:57.772573 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.759631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36780 (local address 127.4.48.252:33601)
0811 12:14:57.759885 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.759891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.770458 (+ 10567us) server_negotiation.cc:407] Connection header received
0811 12:14:57.770994 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.771001 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.771103 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.771186 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.772314 (+ 1128us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36780: BlockingRecv error: recv got EOF from 127.0.0.1:36780 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 12:14:57.797323 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.790305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33500 (local address 127.4.48.253:38551)
0811 12:14:57.790488 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.790493 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.791008 (+ 515us) server_negotiation.cc:407] Connection header received
0811 12:14:57.791224 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.791230 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.791337 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.791434 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.791938 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.793085 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.795064 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.795845 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.795986 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.796428 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.796585 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.796681 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.796888 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.796979 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 12:14:57.805526 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.803415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36786 (local address 127.4.48.252:33601)
0811 12:14:57.803590 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.803596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.804144 (+ 548us) server_negotiation.cc:407] Connection header received
0811 12:14:57.804289 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.804295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.804405 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.804512 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.805299 (+ 787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36786: BlockingRecv error: recv got EOF from 127.0.0.1:36786 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:14:57.819455 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.812911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33508 (local address 127.4.48.253:38551)
0811 12:14:57.813114 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.813124 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.813503 (+ 379us) server_negotiation.cc:407] Connection header received
0811 12:14:57.813676 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.813682 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.813781 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.813865 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.814302 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.815425 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.817359 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.818124 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.818255 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.818716 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.818856 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.818951 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.819097 (+ 146us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.819174 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 12:14:57.827955 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.826196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36796 (local address 127.4.48.252:33601)
0811 12:14:57.826467 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.826473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.826493 (+ 20us) server_negotiation.cc:407] Connection header received
0811 12:14:57.826564 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.826569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.826677 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.826789 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.827702 (+ 913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36796: BlockingRecv error: recv got EOF from 127.0.0.1:36796 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
I20260811 12:14:57.846092 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.839571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33516 (local address 127.4.48.253:38551)
0811 12:14:57.839818 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.839825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.840344 (+ 519us) server_negotiation.cc:407] Connection header received
0811 12:14:57.840552 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.840557 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.840646 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.840749 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.841235 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.842371 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.844341 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.844762 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.844850 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.845331 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.845477 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.845541 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.845725 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.845808 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 12:14:57.853652 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.851585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36804 (local address 127.4.48.252:33601)
0811 12:14:57.851800 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.851806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.852191 (+ 385us) server_negotiation.cc:407] Connection header received
0811 12:14:57.852407 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.852413 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.852487 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.852568 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.853368 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36804: BlockingRecv error: recv got EOF from 127.0.0.1:36804 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:14:57.878651 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.873026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33524 (local address 127.4.48.253:38551)
0811 12:14:57.873163 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.873167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.873470 (+ 303us) server_negotiation.cc:407] Connection header received
0811 12:14:57.873650 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.873653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.873716 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.873803 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.874251 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.875057 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.876948 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.877418 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.877510 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.878019 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.878118 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.878179 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.878344 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.878412 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 12:14:57.886092 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.884034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36814 (local address 127.4.48.252:33601)
0811 12:14:57.884215 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.884220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.884682 (+ 462us) server_negotiation.cc:407] Connection header received
0811 12:14:57.884844 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.884848 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.884926 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.885003 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.885873 (+ 870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36814: BlockingRecv error: recv got EOF from 127.0.0.1:36814 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 12:14:57.928464 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:57.922525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33538 (local address 127.4.48.253:38551)
0811 12:14:57.922725 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.922731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.923214 (+ 483us) server_negotiation.cc:407] Connection header received
0811 12:14:57.923420 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.923425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.923495 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.923575 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:57.924004 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.924979 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.926357 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:57.927130 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:57.927272 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:57.927665 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:57.927828 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:57.927982 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:57.928045 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 12:14:57.928126 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 12:14:57.936393 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:57.934782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36826 (local address 127.4.48.252:33601)
0811 12:14:57.934965 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:57.934970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:57.935208 (+ 238us) server_negotiation.cc:407] Connection header received
0811 12:14:57.935326 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:57.935331 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:57.935417 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:57.935513 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:57.936174 (+ 661us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36826: BlockingRecv error: recv got EOF from 127.0.0.1:36826 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 12:14:58.008536 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.004079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33548 (local address 127.4.48.253:38551)
0811 12:14:58.004295 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.004301 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.004522 (+ 221us) server_negotiation.cc:407] Connection header received
0811 12:14:58.004675 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.004679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.004749 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.004822 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.005169 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.005861 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.007078 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.007464 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.007553 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.007944 (+ 391us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.008042 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.008105 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.008196 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.008257 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 12:14:58.015451 10829 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:14:58.013712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36830 (local address 127.4.48.252:33601)
0811 12:14:58.013912 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.013919 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.014174 (+ 255us) server_negotiation.cc:407] Connection header received
0811 12:14:58.014349 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.014355 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.014445 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.014528 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:14:58.015249 (+ 721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36830: BlockingRecv error: recv got EOF from 127.0.0.1:36830 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:14:58.138707 10853 catalog_manager.cc:1259] Loaded cluster ID: acf8b387d9d849dd9598fb7203336c2b
I20260811 12:14:58.138813 10853 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: loading cluster ID for follower catalog manager: success
I20260811 12:14:58.141422 10853 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: acquiring CA information for follower catalog manager: success
I20260811 12:14:58.143247 10853 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 19691e289d9248dc89a7cf35f075b019: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:14:58.153913 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.149265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.4.48.253:38551)
0811 12:14:58.149515 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.149522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.149869 (+ 347us) server_negotiation.cc:407] Connection header received
0811 12:14:58.150050 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.150054 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.150128 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.150201 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.150534 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.151227 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.152498 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.152911 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.153000 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.153312 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.153407 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.153465 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.153562 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.153623 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 12:14:58.163873 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.158981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36844 (local address 127.4.48.252:33601)
0811 12:14:58.159185 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.159190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.159479 (+ 289us) server_negotiation.cc:407] Connection header received
0811 12:14:58.159612 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.159615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.159805 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.159899 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.160241 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.160884 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.162017 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.162418 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.162503 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.162889 (+ 386us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.163363 (+ 474us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.163478 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.163515 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.163582 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 12:14:58.176062 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.170401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36856 (local address 127.4.48.252:33601)
0811 12:14:58.170557 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.170562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.171099 (+ 537us) server_negotiation.cc:407] Connection header received
0811 12:14:58.171270 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.171275 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.171346 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.171422 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.171942 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.172628 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.174148 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.174567 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.174675 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.175295 (+ 620us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.175410 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.175468 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.175713 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.175790 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:14:58.176833 10708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.169149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33558 (local address 127.4.48.253:38551)
0811 12:14:58.169344 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.169350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.170276 (+ 926us) server_negotiation.cc:407] Connection header received
0811 12:14:58.170547 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.170553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.170639 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.170698 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.171539 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.172641 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.175204 (+ 2563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.175623 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.175759 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.176199 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.176318 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.176411 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.176506 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.176580 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 12:14:58.177548 11092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:14:58.169051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55484 (local address 127.4.48.254:38589)
0811 12:14:58.169734 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 12:14:58.169740 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:14:58.170107 (+ 367us) server_negotiation.cc:407] Connection header received
0811 12:14:58.170361 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:14:58.170366 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:14:58.170583 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:14:58.170759 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:14:58.171540 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.172654 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.175178 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:14:58.175818 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:14:58.175934 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:14:58.176467 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:14:58.176879 (+ 412us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:14:58.176992 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 12:14:58.177173 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 12:14:58.177251 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":558,"thread_start_us":426,"threads_started":1}
I20260811 12:14:58.181433 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10462
I20260811 12:14:58.182539 10568 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:58.288945 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10462
I20260811 12:14:58.319257 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10586
I20260811 12:14:58.320449 10697 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:58.425081 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10586
I20260811 12:14:58.451022 4291 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu pid 10712
I20260811 12:14:58.452138 10814 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 12:14:58.556421 4291 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskXBhqNr/build/asan/bin/kudu with pid 10712
2026-08-11T12:14:58Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (4450 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (9447 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (365653 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST